4D Server 17R6, windows server 2016, 28 processor cores, 180GB ram, 80 GB cache
I have a large (170GB), busy (200+ users) database that I’m struggling with major slow-downs.
To help fix the slowdowns: I’ve recently migrated a bunch of various task processing from cooperative, to Preemptive threads. The slowdowns have in some ways become more profound.
I have a table of [DB_Tasks], where I create records of work that needs to be done. I have a task processing routine that processes that tasks: and logs into the completed record how much processing-time it took (based on PROCESS PROPERTIES procTime). I can see that some tasks are taking too much time: but since moving from cooperative to preemptive, the amount of processing time seems to have greatly increased.
The reason that I moved the tasks to a preemptive thread: was to lighten the load on the main core of the server: I don’t really care if the processes take “more processing time” as long as it doesn’t impact the users.
However: from what I’m seeing: even though the processes are now in preemptive: the database performance becomes very bad when I have multiple preemptive processes that get busy. Today, it became so bad at one point that I had to restart the 4D server: it became uselessly un-responsive. (at the time, there were probably 10+ preemptive tasks fighting to do work: if they can’t complete in a timely manner: they spawn more)
(My tasks aren’t doing anything that I think should take that much time: scanning records that should already be in the database cache: comparing values to other records: and adding/modifying/deleting a nominal amount of records based on the results. Tested on a local machine: the same processing might take a few seconds: but my server tasks (when performing badly) are taking 30+ seconds. )
I would dearly like to add Debug Logging to my preemptive threads to help diagnose the issue: but that does not seem to be available in Preemptive. (major bummer)
WHY would preemptive threads compete with my primary Server core performance – and cause server speed degradation?
I’m entering that terrible state of desperation.