Storage is suddenly empty

We’re using Storage on a v17.1 Server to replace <>interprocess variables and track things across processes. We have a collection in Storage to keep track of the status of jobs to do in workers (some are preemptive). When each task is done, we remove it from the collection.

Seemingly at random Storage will come up empty - all the objects and collections inside are gone.
This has only happened compiled, client/server (on the server), with some preemptive threads.

Can anyone think of a way that all the objects in Storage would be blown away?

v17.1
4D Remote 32 bit
4D Server 64 bit

Also tested
v17R5 build 237893
4D Remote 64 bit
4D Server 64 bit

Windows 10

Thanks,

How are you detecting that storage is empty?

Could you show us the code?

The first indication is the Runtime error on the server when a worker tries to access something in Storage.

We also put this in a method with execute on server attribute checked:
<code 4D>
SET TEXT TO PASTEBOARD(JSON Stringify(Storage;*))
</code 4D>

Call the method On Timer every half second from the client.

Yes, we had the same problem until v17R4. In seems, that happens after an uptime of at least one day.

What’s the runtime error message?

What’s the schema for the storage object?

Does the storage object store only scalar values? Or are there references also stored?

The error message is:

Expecting a shared object or shared collection.

Use (Storage.cache)

Storage.cache existed after On Server Startup, before we started running our worker:

{
“dl”: {
“ID_Company”: “REL140000000001”,
“company”: “Relevant Partners, L.P.”,
“owner”: “Relevant Partners, L.P.”,
“country”: “USA”,
“text”: “Relevant Partners, L.P.”,
“demo”: true,
“devPhone”: “781-250-4000”,
“ID_Contact”: [
“REL150000000014”,
],
“contact”: [
“Hays, Jim”,
],
“ID_prefix”: “”
},
“grid”: {
“dateFormat”: “7-”
},
“dlg”: {
“bttnRightMargin”: 20,
“leftMargin”: 28,
“topSuppInstMargin”: 59,
“topMargin”: 25,
“bttnBottomMargin”: 15,
“dlgBttnHeight”: 20
},
“winRefs”: [],
“cache”: {
“triggersOn”: true,
“cacheList”: [
{
“active”: true,
“name”: “Cache_CoGrpLink”,
“description”: “Company - Teams & Offices”
},
{
“active”: true,
“name”: “Cache_CoPrimaryContact”,
“description”: “Company - Primary Contacts”
},
{
“active”: true,
“name”: “Cache_CoRecentInteraction”,
“description”: “Company - Recent Interactions”
},
{
“active”: true,
“name”: “Cache_CoRoles_DT”,
“description”: “Company - Deal Team Roles”
},
{
“active”: true,
“name”: “Cache_CoRoles_PT”,
“description”: “Company - Partner Team Roles”
},
{
“active”: true,
“name”: “Cache_InteractionOrg”,
“description”: “Interaction - Linked Companies, Partners, Funds”
},
{
“active”: true,
“name”: “Cache_CtAddress”,
“description”: “Contact - Active Address”
},
{
“active”: true,
“name”: “Cache_PtnrRoles_PT”,
“description”: “Partner - Mailing List Roles”
},
{
“active”: true,
“name”: “Cache_CA_SignedAmts”,
“description”: “Fund Txn - Signed Amounts”
},
{
“active”: true,
“name”: “Cache_PtnrCapAcct”,
“description”: “Partner - Capital Account”
},
{
“active”: true,
“name”: “Cache_InvCapAcct”,
“description”: “Investor - Capital Account”
},
{
“active”: false,
“name”: “Cache_FundCapAcct”,
“description”: “Fund - Capital Account”
},
{
“active”: false,
“name”: “Cache_FMRCapAcct”,
“description”: “Fund Manager - Capital Account”
},
{
“active”: false,
“name”: “Cache_FMRStats”,
“description”: “Fund Manager - Statistics”
},
{
“active”: false,
“name”: “Cache_FundPortfolio”,
“description”: “Fund - Portfolio”
},
{
“active”: true,
“name”: “Cache_PCOPortfolio”,
“description”: “Portfolio Company - Portfolio”
},
{
“active”: false,
“name”: “Cache_PCOFundCmmt”,
“description”: “Portfolio Company - Fund Commitment”
},
{
“active”: false,
“name”: “Cache_RealEstate”,
“description”: “Company - Real Estate”
},
{
“active”: false,
“client”: “Kirkland”,
“name”: “Cache_UDF”,
“description”: “Fund - User Defined Fields”
},
{
“active”: false,
“client”: “TA”,
“name”: “Cache_CtRoles_FR”,
“description”: “Contact - Fund Raising Roles”
},
{
“active”: false,
“client”: “Carroll”,
“name”: “Cache_CarrollOrg”,
“description”: “Property - Carroll Custom Fields”
}
],
“port”: {
“scope”: 1000000
},
“fmr”: {
“scope”: 1000000
},
“ptnr”: {
“scope”: 1000000
},
“inv”: {
“scope”: 1000000
},
“fund”: {
“scope”: 1000000
},
“activity”: [],
“throttleSeconds”: 0
},
“webApps”: [
{
“appName”: “Reports”,
“code”: 0
},
{
“appName”: “Monitoring”,
“code”: 1
}
],
“user”: {
“name”: “Admin”,
“initials”: “ADMI”
},
“FA”: {
“IRR_Use_Calls_Rcvd”: false
},
“app”: {
“quit”: false
}
}

We’re not storing any references in Storage.

So you know that Storage may become an empty object.

But where does the runtime error occur?

What happens if you set up an error handler – that logs to a file –
the following information:

<https://doc.4d.com/4Dv17R5/4D/17-R5/GET-LAST-ERROR-STACK.301-4127388.e
.html>GET LAST ERROR STACK>
These
<https://doc.4d.com/4Dv17R5/4D/17-R5/System-Variables.300-4128323.en.ht
l#2882166>system error variables>:

Error method
Error line
Error formula

Again:

the Storage object is empty. So if you want to access your sub-objects, they are not there.

So there is no error and nothing to catch.

: Christian SAKOWSKI

…there is no error and nothing to catch.

There appears to be a runtime error on the server’s Storage object. See Jim’s https://forums.4d.com/Post/EN/31352013/1/31384371#31353957post #3>.

And the error message appears to be “Expecting a shared object or shared collection.” See Jim’s https://forums.4d.com/Post/EN/31352013/1/31384371#31369439post #6>. That is possibly error # https://doc.4d.com/4Dv17R5/4D/17-R5/Object-Notation-Analysis-Errors-10737-10701.300-4127423.en.html-10722>.

The next step to sleuth the Storage object problem is to locate at the code line on which the error occurs, and see what can be gleaned.

I’ve submitted a small database that reproduces the problem to 4D tech support. I’ll let you know what comes of it.

Jim

Hi Jim,

can you upload the database here in the forum? Would be very interesting.

I’ve uploaded the database. Have fun!
(And feel free to suggest better methods!)

https://forums.4d.com/4DBB_Main/x_User/1162299/files/31580502.zip

More testing with the sample database:

17.2HF1
Doesn’t show empty Storage, but it crashes with repeated tests.

17R6 build 241080
Does not seem to have any problems so far.
Except for that we aren’t quite ready to go 64-bit on the 4D Remote side.

Hi Jim,

I encountered almost the same issue in spring, except it appeared in 4D Desktop, but through a component.

I was trying to use in a component a collection inside an object in Storage (with push & remove), and tested in 64bits compiled preemptive process.
A some point, Storage was just gone… empty (never at the same amount of iterations).
But it worked fine in nonpreemptive processes or interpreted (with the same component)

I fixed my code by not using collection, but instead I used associative arrays, which was finally faster in my case.

So, definitively something odd with collections in Storage, in preemptive process.
I talked about that at the 4D Tours (Paris) to several member of 4D dev staff and apparently, at that time, it seems I was the only one experiencing that king of behavior (not much people doing preemptive stuffs in v17 components)
I didn’t have the time then to explore further nor to report.

That seems to be the answer.

I was using a collection with calls like this:

Storage.cache.activity.push($item_o)
Storage.cache.activity.remove($index)

Changing it to just using an object with named items whose value is the status.
Storage.cache.activity[$item_t]:=$parms_o.status

I can now run all my caches in v17.2hf1, even overload the my worker queues and it doesn’t blink.

Thanks!

Jim

Happy to help.

Keep in mind you can use OB REMOVE to delete an item of your list and use the For each statement to loop on each item.