4D View Pro sheet size

Hello everybody,
I need an advice for a database problem related to the v18 4D View Pro.

As said in another post (https://forums.4d.com/Post/IT/29978300/1/29978301#29978301) my company massively relies on 4D View sheets.

One of our simplest sheets may be like this
[]33939583;“Example of a sheet”[/]
With text, borders, formulas, different backgrounds, different fonts, about 30 columns and 100 rows. Other sheets may have also images.

This 4D View document, exported as 4PV has a size of 27kB.
This 4D View document, stored in a compressed blob has a size of 3kB.

If I convert this sheet from View to View Pro the stringified JSON has a size of 243kB. Because there are a lot of redundant attributes for these 30x100 cells.
The 4VP file has the same size of 243kB.
If I put the object in a blob, the resulting blob has a size of 433 kB.
If I put the object in a blob and I compress it, the resulting blob has a size of 10 kB.
The excel converted file has a size of 16kB.

[]33939591;“Your comment here…”[/]

Currently we store in our main v17 database about 700 000 compressed blobs containing a 4D View sheet.
I really loved the idea to use objects with all their features, but in these conditions it’s not sustainable.

Do I really have to store all the 4D View Pro objects in compressed blobs (tripling anyway the space needed in comparison to the old 4D View)? Do you have other ideas?

Thanks to whoever contributes

Just for test: what is the size of the 4VP (not the 4PV !) document zipped :?:

Only 5 kB!

Ok, so now, you have just to ask 4D to include a new command to zip the objet in the database :mrgreen: :lol:

https://blog.4d.com/zip-unzip-files-and-folders-with-these-commands/A start point> :?:

Thank you Manuel for your suggestion.
I don’t know if it’s a good idea to handle over 800000 zip files (I was wrong when I said 700000) in the filesystem instead of using the (very practical) 4D database.

The majority of these sheets is stored for historical reasons (we have to keep data at least 10 years) so a possible way to approach the problem is to keep intact these old View blobs and convert them to View Pro only when/if necessary.

The problem is not solved because within 10 years the database will be full off “space consuming” objects.

Hello Matteo,

You will find enclosed a little database that I made to test the difference of size of 4PV vs 4VP documents.

There is a sample in the resources folder you can load for testing that shows those values, but it will be interesting to see the results with your own documents.

4D View document : 2 948 bytes
UTF8 stringified 4D View Pro document : 8981 bytes
Compressed blob containing the stringified 4D View Pro document : 1010 bytes

Those results are coherent and show the differences between a non compressed binary document (4D View), a non compressed text document (that is necessarily bigger by nature) and a compressed text document (plain text is known to compress very well).

https://forums.4d.com/4DBB_Main/x_User/3889/files/34021997.zip

I tested my example View file with your application:
size of stringified doc: 242955
size of compressed doc :4503

The results are similar to those in my first post.

Here you can find attached the 4PV of the first post: https://forums.4d.com/4DBB_Main/x_User/695643/files/34023303.zip

When you convert it to View Pro, one of the reasons of the increased size of the object is the style of the cells repeated thousand of times. With the old View there is no problem.

I am a little bit puzzled by your answer did it mean that we should NOT use Object field to store 4D View Pro object zone in a 4D database :?: :doubt: :-?

: Manuel PIQUET

I am a little bit puzzled by your answer did it mean that we should
NOT use Object field to store 4D View Pro object zone in a 4D
database :?: :doubt: :-?

Same feeling !! And I assume that the answer would be the same for View Pro and Write Pro.

I am a little bit puzzled by your answer did it mean that we should NOT use Object field to store 4D View Pro object zone in a 4D database

You can do it of course but if you use a compressed blob it will of course use less room on disk at the cost of a little more code. I think you will have more benefit to do it if like Matteo you have plenty of converted 4D View documents that converts in very big documents due to the way they were built in 4D View. In this case there are plenty of redundant (but mandatory) informations generated by the converter and it’s a good idea to store compressed blobs that will use half of the space of the original 4D View documents instead of plain text or objects that will use 100 times more space.

For regular documents created directly using 4D View Pro, we recommand to store them as objects because you will have all the benefits of storing those documents in a database, and they will have very reasonnable size.

The big benefit is that you can use Query by attributes to find 4D View Pro documents that contains some specific value.

Of course if you don’t need to Query in your 4D View Pro documents (or if you handle the search criteria your self outside the document) and if total size of the database is important in your case then you may choose to use compressed blobs.

I was puzzled, now I am shocked… :-o

We should embeded objet into BLOB to compress it to store it in 4D database… :-?

It is REALLY NOT THE SAME !!!

Objects are managed by ORDA , REST, etc…

BLOB NO :!: :!: :!:

: Francois MARCHAL

For regular documents created directly using 4D View Pro, we
recommand to store them as objects because you will have all the
benefits of storing those documents in a database, and they will have
very reasonnable size.
Do I understand correctly : only converted documents have a big size ?
If so, is it possible to do something after conversion to reduce the size ?

Do I understand correctly : only converted documents have a big size ?

Not all the converted documents, but in the document that Matteo sends, all the cells have been individually formatted with a color that changes depending on the value. This leads to a very verbose but mandatory conversion in the 4D View Pro document, but not all the 4D View documents are like that.

If you well know how your documents are created, you can imagine to reduce the size by using for example a stylesheet instead of modifying directly the cell style.

Dear Francois,
what do you mean when you say “with a color that changes depending on the value”? The background color should be static.

Could it be possible that you do that automatically (during the conversion) for us :?: :pray:

If a cell use a specific style that is already use in the same document then create automatically a new stylesheet and use it instead of apply the same style for each cell individually :idea:

what do you mean when you say “with a color that changes depending on the value”? The background color should be static.

Browsing your document, I see that most of your cells have the following style

formatter : “General;[#FF0000]-General;[#0000FF]General”

that means that the text color will vary depending the value. Black for positive values, red for negative values and blue for zero

: Francois MARCHAL

that means that the text color will vary depending the value. Black
for positive values, red for negative values and blue for zero

This is a good example because this is the default setting for each cell in 4D View document :!:

So I really think you should add automatically at least a new stylesheet only for this particular case.

See this screenshot from the setting of a cell from an empty 4D View

[]34025839;“Default setting…”[/]