Ausführungszeit von VERIFY CURRENT DATA FILE

Product :4D - 4D Server, compiliert
OS : Windows
4D : v16.2 (214903)

Unsere 4D Applikation führt auf dem Server täglich um Mitternacht einen DB Verify aus, um die Konsitenz und Integrität der Daten sicherzustellen. Normalerweise dauert der Verify ca. 5 Minuten. Es kommt aber sporadisch vor, dass sich der Verify über 30 Minuten Zeit nimmt.

Die DB hat etwas über 22Mio Records, in 128 Tabellen, mit 766 Feldern, davon 389 indiziert. Wir generieren pro Tag ziemlich konstant so ca. 50’000 - 80’000 neue Records. RAM & CPU usage waren normal während des verlängerten Verifys, auch haben wir keine Hinweise, dass an diesen Tagen die DB irgendie übergebührlich benutzt wurde.

Anbei noch ein Plot mit Number of Records und der täglichen Ausführungszeit des DB Verifys.
[]30292328;“NrOfRecords vs. DB Verify Duration”[/]

Weiss jemand Hinweise, woran diese sporadisch sehr langen Verify Zeiten liegen können?

Grüsse aus der :ch:
Simon

Zur Vollständigkeit habe ich noch die InfoReports und Verify Resultate angehängt.

https://forums.4d.com/4DBB_Main/x_User/18134443/files/30293327.zip
https://forums.4d.com/4DBB_Main/x_User/18134443/files/30293332.zip

Und hier noch die Memory usage (auch nichts unauffälliges)
[]30294476;“Memory vs. Veriy Duration”[/]

Hi,

(answer in english…)

There are two Attentions in the reports raised since May 18:

Attention : The computer has not been restarted for more than 3 weeks.

Attention : Dump : CurrencyXchanger EE [SBB-PROD].exe.dmp
(Size: 0 KB) Modif: 2019_06_17__08_20_05

In the Verify reports:
May 17: (less than 5 minutes) :
<start_timer time=“2019-06-17T22:47:36+02:00”/>
<stop_timer time=“2019-06-17T22:51:15+02:00” success=“true” user_canceled=“false”/>

May 18: (37 minutes) :
<start_timer time=“2019-06-18T22:46:23+02:00”/>
<stop_timer time=“2019-06-18T23:23:37+02:00” success=“true” user_canceled=“false”/>

May 19: (more than 38 minutes) :
<start_timer time=“2019-06-19T22:45:43+02:00”/>
<stop_timer time=“2019-06-19T23:23:15+02:00” success=“true” user_canceled=“false”/>

There is nothing obvious in the cause of this large slowdown of the VERIFY CURRENT DATA FILE from the numbers in the Info Reports: 2 users connected, size of the data files almost the same, the Used cache size is larger:

Report_2019_06_17_22_51_02:
Data File:
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DD”
Size: 17 554 240 KB Modif: 2019_06_17__22_45_28

Data File: (index)
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DIndx”
Size: 4 901 632 KB Modif: 2019_06_17__19_36_54

Used cache Size : 17 027 548 KB

Report_2019_06_18_22_54_03:
Data File:
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DD”
Size: 17 605 568 KB Modif: 2019_06_18__22_46_14

Data File: (index)
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DIndx”
Size: 4 914 176 KB Modif: 2019_06_18__19_31_13

Used cache Size : 19 222 766 KB

Report_2019_05_19_22_47_24:
Data File:
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DD”
Size: 17 676 032 KB Modif: 2019_06_19__22_44_19

Data File: (index)
“D:\CXR_Data\CurrencyXchanger EE [SBB-PROD].4DIndx”
Size: 4 929 280 KB Modif: 2019_06_19__18_12_25

Used cache Size : 19 740 620 KB

There are still plenty of memory available to increase the Cache size (32 000 MB) as there is 128 GB of RAM.

I suggest to increase the size of the Cache (to 40 000 MB for example), to see if this is reducing again the time of the VERIFY CURRENT DATA FILE.

The command SET CACHE SIZE (introduced in v16) can also increase temporarily the Cache size before the verification.
https://doc.4d.com/4Dv16/4D/16.5/SET-CACHE-SIZE.301-4226249.de.html

HTH,
Thomas S.

Hi Thomas,

Many thanks for the quick response.

We will observe tonight, whether the verify time goes back to normal 5mins. If not, we will experiment with the cache size and report back…

Best regards,
Simon

In the meantime we did some tests.

Same datafile (copy of 4DD datafile, copied while the service was stopped), sampe 4D application, but different machine (we do not like to test&play on production systems).

Settings:

  • same data file (copy of 4DD datafile, copied while the service was stopped)
  • same 4D application
  • different machine (one of our 4-core test VMs, with 8GB ram)
  • Cache set to 8GB

Findings on test machine:

  • Verify needed 1h20min (expected, the machine is considerably slower)
  • Verify did at maximum use about 2GB RAM (this is an indication that the cache size is not the limit)
  • then executed a compact. Data file had about the same size
  • Verify after compact needed 20mins (I suspect this would correspond to ~4-5-min verification time on the production systems, very much the same as before the verification time drastically increased)

Question:

  • Is this expected that a compact will have such drastic impact on verify time?
  • What changes/modifications/alterations can lead to such an increase in verify? Is this an indication that something is ~“broken”~in the datafile?
  • Is a regular execution of compact needed? If yes what “compact strategy” do you suggest?

Best,
Simon