FWD » Core Development » Runtime Infrastructure

• LegacyServiceWorker threads, which manage incoming requests
• MsaTask threads, which perform the actual work

Currently, a LegacyServiceWorker launches an MsaTask in a blocking manner and waits for that task to complete. While this model works, it effectively ties up the LegacyServiceWorker thread for the duration of the task, limiting overall throughput under load.
If we were to make this interaction fully asynchronous, allowing the LegacyServiceWorker to dispatch the task and immediately return to handling other requests, we could improve thread utilization and potentially increase total system throughput.
I don't yet have concrete numbers on the expected improvement, but architecturally this feels like a logical next step, especially as we continue to scale.

I have planned #6673 which I believe is the same idea. I would be happy for you to work on that task. My opinion is that a thread-safe blocking queue is a very good implementation for this purpose.

Constantin, please correct me if I am not accurate here, but I think this already happens with PASOE.
AppServerConnectionPool has BlockingQueue of worker service threads which in turn execute the MSA tasks.

What I am talking about is using the current infrastructure that relies on Futures to execute tasks, but is doing it in a blocking manner, which defeats the purpose of using the Futures.

I believe for classic this does not happen and on agent pool exhaustion, the requests are refused.

My idea was not to have a queue for getting a thread but a queue for the jobs. Also the idea is to do almost nothing in the Jetty context other than add the request to the queue and then continue processing. This affects both clasic and PASOE, I think.

Greg Shah wrote:

My idea was not to have a queue for getting a thread but a queue for the jobs. Also the idea is to do almost nothing in the Jetty context other than add the request to the queue and then continue processing. This affects both clasic and PASOE, I think.

If by 'jobs' you mean the MsaTasks in PASOE, running the programs and procedures on the appserver, then this already happens with the use of a ThreadPoolExecutor that has a LinkedBlockingQueue for tasks.

I think we do more on the Jetty side than I want. I'm looking for a lower level transfer of the request. Also, I want this improvement in both classic and pasoe modes.

Sure, I will look into this as well.

I investigated the appserver task queuing path and found a potential optimization in the worker wake up mechanism. In the current implementation, enqueuing work notifies all waiting worker threads, which can create unnecessary wake up churn under load. Multiple idle threads are awakened for a single unit of work, only one actually acquires the task, and the rest immediately return to waiting. That adds extra context switches, lock contention and queue pressure without increasing useful parallelism.
I experimented with changing the notification strategy so that only one idle worker is signaled when a task is posted. The expectation was that this would reduce contention in the handoff path and improve throughput at higher concurrency. In practice, the benchmark results did not improve, which suggests that the task wake up mechanism is not that demanding.

Are you testing it on a system with hundreds of sessions that are being heavily hit? The more threads and load, the more likely this contention is an issue.

Yes, I am using the load tests from the multi tenant app, hundreds of concurrent users.