All Questions
Tagged with disruptor-pattern lmax
54 questions
0
votes
0
answers
21
views
Tracing in LMAX disrupter processing
I am looking for options to be able to fetch and track logs in the flow of handlers processing in LMAX ring buffer, i.e, for an particular message posted into ring buffer, get all its respective logs ...
0
votes
1
answer
78
views
How to wait for all Disruptor messages to be consumed in a test case
I'm trying to unit test an application which publishes/consumes messages using the Disruptor ring buffer. In the test I would like to
Configure the Disruptor and it's EventHandlers
Publish messages ...
- 27.8k
0
votes
0
answers
316
views
How to instantiate LMAX Disruptor for sequential processing
Version 3 of the LMAX Disruptor deprecated the constructor that takes an Executor. So the following gives a compile warning:
Disruptor(eventFactory, bufferSize, Executors.newSingleThreadExecutor())
...
- 131
0
votes
1
answer
74
views
How to convert ReentrantReadWriteLock logic to LMAX Disruptor with barriers
I have a shared collection an ArrayList and also i use a ReentrantReadWriteLock lock to secure the entering on a critical area from different threads. My threads are three a writer,read,delete thread. ...
- 319
0
votes
0
answers
185
views
Limit number of active threads in LMAX Disruptor(RingBuffer)
I have multiple consumers of different types that can be run in parallel but I don't want all of them to run in parallel at the same time because I don't have that many CPU cores and performance would ...
- 3,161
0
votes
1
answer
522
views
Using lmax Disruptor (3.0) in java to process millions of documents
I have the following use-case:
When my service starts, it may need to deal with millions of documents in as short of a burst as possible. There will be three sources of data.
I have set up the ...
- 2,148
2
votes
1
answer
839
views
AttributeError: Can't get attribute 'journalerReader' on <module '__mp_main__
I tried to implement Lmax in python .I tried to handle data in 4 processes
import disruptor
import multiprocessing
import random
if __name__ == '__main__':
cb = disruptor.CircularBuffer(5)
...
- 7,261
2
votes
0
answers
330
views
Business logic imp in LMAX architecture in python
I am trying to implement disruptor in Lmax architecture .
As you know in lmax architecture we have a ring buffer for creating a queue to process data .
Here you can see the structure of that :
I have ...
- 7,261
0
votes
1
answer
248
views
LMAX Disruptor remainingCapacity equals 0, even before some time waiting
I added data to Disruptor by calling tryPublishEvent function.
After I wait 40 second and tried to check unprocessedDataCount by following calculation:
long ringBufferUnProcessedCount = disruptor....
- 156
1
vote
1
answer
970
views
How is LMAX Disruptor garbage collector friendly
I am trying to understand how lmax disruptor is GC friendly. I see that the event objects on the ring are re-used, however, the "data" that is "set" on these objects are on the ...
- 2,306
0
votes
1
answer
480
views
Disruptor behavior - Drain full buffer before consuming new data
I have the following scenario.
We were load testing our application in Cloud on K8s. Our inbound messages are coming over Kafka and we are writing back to Kafka. Our architecture is such that where ...
- 574
2
votes
1
answer
676
views
Using LMAX Disruptor for event handling reduces performance compared to Netty Handlers
I have written a Netty HTTP Server.
I am figuring out right way to handle Http requests
Options: (1. Netty Handlers 2. LMAX Disruptor).
I have read it somewhere that LMAX Disruptor are good for ...
- 461
3
votes
1
answer
1k
views
LMAX Disruptor in Python
Is there already an implementation (or at least binding to) Python3?
I can see that an original implementation is in Java on Github, but I do not see anything for Python3.
- 1,983
1
vote
0
answers
296
views
LMAX Disruptor - sequential & parallel database/service calls on event handling
I a have a workflow (different tasks) that process data from Kafka/Redis streams. Some tasks are sequential & some parallel - on the high leve tasks fall into, verification -> computation -> ...
- 2,902
2
votes
1
answer
517
views
Why lmax disruptor architecture use 2 disruptor?
In " lmax disruptor architecture design" it is showing that , they are taking input and enqueing it in input disruptor and there are multiple event handlers like journaling ,un-marshalling , ...
- 106
-1
votes
1
answer
345
views
LMAX's disruptor. How to define multi EventHandlerGroup?
I want to handle different event with multi-EventHandleGroup?I can't find example with that.thanks...
it's the code:
executorService = Executors.newFixedThreadPool(threadPoolSize);
disruptor = ...
- 1
1
vote
1
answer
172
views
LMAX Distruptor Partition and join batch
So currently I have a Executor implementation with blocking queue and the implementation specific is like, I have list of items per request and I divide them into partitions each partition is then ...
- 196
0
votes
0
answers
315
views
Is the lmax disruptor a good architecture choice where there are several consumers all of which perform the same task and rely on some local cache?
My use case is this -
Several events are produced and dispatched onto a threadpool.
Each event is processed separately and the output of the processing isn't needed for some reconciliation later. (So,...
0
votes
1
answer
891
views
LMAX Disruptor Producer Incorrectly Wraparound + Overwrite Before Consumer Completes Read
I am recently introduced to the LMAX Disruptor and decided to give it a try. Thanks to the developers, the setup was quick and hassle free. But I think I am running into an issue if someone can help ...
- 11
-1
votes
1
answer
481
views
how to fulfill the ringbuffer with data from my hashmap lmax disruptor
Actually, after reading and using a simple example of disruptive I can't find how to where should I fulfill my the ring buffer with data from my hashmap already fulfilled with data from ...
1
vote
1
answer
399
views
how to push list to lmax disruptor in the eventclass
I need to store and get a huge data from mongodb so Im asked to receive and store data using lmax disruptor I have passed few days looking to a simple tutorial on the lmax github account, but I didn ...
0
votes
1
answer
400
views
LMAX Disruptor - Maintain order of events
I have an application that loads time series data from various files. The application opens one thread per file to load the data in parallel. The records in the files are ordered but I need to deliver ...
- 1
4
votes
1
answer
1k
views
Disruptor vs. Reactive architecture for middle-frequency trading system
I'm trying to choose an appropriate architecture for a middle-frequency trading system I'm working on. Currently, I receive messages from Web Socket or Rest and process them right there. Sometimes it ...
- 25.3k
1
vote
2
answers
1k
views
Disruptor Journaling Implementation
In Martin Fowler's write-up of the LMAX-disruptor architecture, he says:
The journaler's job is to store all the events in a durable form, so
that they can be replayed should anything go wrong. ...
- 68.5k
3
votes
2
answers
3k
views
Why is disruptor slower with smaller ring buffer?
Following the Disruptor Getting Started Guide, I've built a minimal disruptor with a single producer and single consumer.
Producer
import com.lmax.disruptor.RingBuffer;
public class ...
- 16.2k
1
vote
1
answer
355
views
LMAX Disruptor Dependency Graph/Gating with SequenceBarrier
Goal
I'm trying to create a dependency relationship between handlers that's somewhat circular, and I can't quite figure out how to get it right. What I want to achieve is a variation of producer ->...
- 1,904
3
votes
2
answers
1k
views
Lmax Disruptor, many consumers - how make consumer take only messages of particular kind and proceed independently?
I plan to have many parallel consumers in my disruptor.
I need each consumer only consume messages that are meant for them.
For instance, I have messages of types A, B, C and I have buffer like
...
- 9,371
2
votes
1
answer
97
views
"Finally" or "Do Last" for LMAX Disruptor?
I'm trying to write a wrapper for the LMAX disruptor, and I'm wondering if there is a mechanism to add an event handler that executes at the end of the disrupter's flow.
Is there a way to do this?
- 55.6k
1
vote
0
answers
159
views
Cannot get bean of Service with annotation @resource when Spring MVC Integration Disruptor
the code as below:
@Controller
public class MapEventHandler implements EventHandler{
@Resource
VisitorMacService visitorMacService;
@RequestMapping("/maptest")
@ResponseBody
public String maptest()...
- 11
2
votes
1
answer
377
views
Pause the processing inside lmax disruptor?
I would like to do a performance test using Lmax Disruptor:
Configure dusruptor
Somehow "pause" the processing
Add many messages to RingBuffer
"Unpause" the processing
This way I could measure ...
- 9,371
0
votes
1
answer
482
views
Passing data between subsequental Consumers in LMAX Disruptor (from unmarchaler to business logic)?
As explained in https://martinfowler.com/articles/lmax.html, I would need to process my RingBuffer's events first with Unmarchaler and then with Business Logic Processor. Suppose it is configured like ...
- 9,371
2
votes
1
answer
312
views
Will LMAX Disruptor work on Android?
Anyone had any experience with running LMAX Disruptor on Android? Will it work and are there any compatibility issues?
- 108
2
votes
1
answer
523
views
LMAX Disruptor Timeout for the EventHandler onEvent
I have a ring buffer and an event handler. This is used as an inbound buffer of a messaging server.
Inside my even handler, it does some AMQP service invocations and some times due to that call ...
2
votes
1
answer
757
views
reactor lmax thread dump
Trying to understand below (partial) stack trace. Using spring's project reactor 2.0.4.
I got task scheduler threads that are supposed to create tasks and allocate tasks for worker threads. The ...
- 97
2
votes
1
answer
920
views
Null Pointer exception while using TimeoutBlockingWaitStrategy in disruptor
I was working with the workerpool example and was trying different WaitStrategies. When i try the TimeoutBlockingWaitStrategy, i am getting error. Here's the program and the call stack.
package org....
- 508
8
votes
1
answer
5k
views
LMAX Disruptor - what determines the batch size?
I have been recently learning about the LMAX Disruptor and been doing some experimentation. One thing that is puzzling me is the endOfBatch parameter of the onEvent handler method of the EventHandler. ...
- 4,782
0
votes
2
answers
905
views
disruptor one event handler stalls other event handlers
I am running a disruptor instance with following event handler:
int NUM_EVENT_PROCESSORS = 5;
executor = Executors.newFixedThreadPool(NUM_EVENT_PROCESSORS);
EventFactory factory = new EventFactory()...
- 3,318
4
votes
2
answers
519
views
Writing with a single thread LMAX
I've got introduced to LMAX and this wonderful concept called RingBuffer.
So guys tell that when writing to the ringbuffer with only one thread performance is way better than with multiple producers......
- 11.3k
2
votes
0
answers
830
views
Usage of executor service in Lmax Disruptor
I am evaluating the Disruptor library for a data processing application platform. Specifically I would evaluate against Akka actor framework and plain Java executor service, fitting to our problem ...
- 661
0
votes
1
answer
349
views
Disruptor park/halt several EventHandlers when exception occurs
We have run into a high CPU usage situation when one of our EventHandlers broke.
Let's say we have several consumers (EventHanlders), that are configured to run sequentially over the buffer. If the ...
- 101
1
vote
1
answer
844
views
how to find a Lmax Disruptor diamond(one producer 5 consumer 1 conclude)example?
i find the user guide of Lmax disrupter in github is very simple, now i have a problem with one producer and five cosumer, after that i need to conclude the result of the consumer, is there any demo, ...
- 87
2
votes
2
answers
1k
views
Getting the current number of messages in the ring buffer
I am using Spring's Reactor pattern in my web application. Internally it uses LMAX's RingBuffer implementation as one of it's message queues. I was wondering if there's any way to find out the current ...
- 603
0
votes
1
answer
2k
views
In an LMAX disruptor like pattern, how do you handle a slow consumer?
I have a question on what to do in a case of a slow consumer in a lmax disruptor like ring buffer that has multi producers and a single consumer running on x86 Linux. With an lmax like ring buffer ...
- 3,336
9
votes
1
answer
2k
views
LMAX Replicator Design - How to support high availability?
LMAX Disruptor is generally implemented using the following approach:
As in this example, Replicator is responsible for replicating the input events\commands to the slave nodes. Replicating across a ...
- 10.8k
0
votes
1
answer
203
views
Maximum number of disruptors in an application
Using LMAX Disruptor,we have observed that if we use like 5-10 disruptors together in an application (sort of like a chain of disruptors with every disruptor having one consumer on it performing a ...
- 69
0
votes
2
answers
1k
views
I don't see the performance improvement of Disruptor
I know my question goes against the very fundamental claim of Disruptor API. But as I was learning about it, I wrote a program to replace my 1P-1C usecase where I was using ArrayLinkedBlockingQueue. ...
- 3,443
1
vote
1
answer
553
views
LMAX Disruptor Timeout EventHandler
We have a system using Disruptor framework, it has five registered stages implementing EvenetHandler.
The stages work in Sequence, so a request can only move to second stage when the first stage is ...
- 11
4
votes
1
answer
1k
views
Monitoring the LMAX Disruptor
How to monitor the LMAX Disruptor? Say I have 3 ringbuffers and wish to provide a ui that will give me information of the ringbuffers.
- 2,902
1
vote
1
answer
2k
views
Using disruptor in the Java Servlet and handling multiple events [closed]
I am using the LMAX disruptor in my web application which takes the http request parameters and handle them to the ringbuffer. 3 event-handlers handle and process data, the final one, saves it to the ...
- 2,902
1
vote
3
answers
1k
views
LMAX Disruptor: Must EventHandler clone object received from EventHandler#onEvent
I have an application with Many Producers and consumers.
From my understanding, RingBuffer creates objects at start of RingBuffer init and you then copy object when you publish in Ring and get them ...
user1565007