Modify .4dm format for easier editing in text editors

I was responding to this;

Which lead me to assume that there is a setting to control this. If there is such a setting, how do we use it?

Pete,

You can write your code in an external editor without the token:

Then look at it 4D:

If you make a modification to the method (eg a comment or even just a blank line or space) and save the method. Then 4D will add the token for you.

I hope that makes sense now.

Wayne

Hi Wayne. Gotcha. That does make sense. Thanks.

1 Like

so if you’re using a vcs, don’t commit until you re-open the project in 4D, correct? otherwise, you’re going to commit the change in the source, but then the source is going to be changed by 4D again to insert the tags? that will be entertaining if you’re working with someone, or several someones, because at least one of them is going to edit a file for something that is minor and push the commit without reopening the project in 4D, first.
that also means that your source is going to have tags in it whether you like it or not, right?
is 4d also going to redo the indenting when it parses the file to add the tokens? are there any other edits that it will be making to the source files?
it would be nice if we could have our source as-is. if 4d is going to parse/precompile/whatever, it could save the result with a separate suffix, which would both keep the source from being polluted and would give the developer an option for ignoring the tokenized files in or to include the tokenized files but have git diff ignore the whitespace changes.

As a workflow, I would recommend loading the files into 4D and starting the compiler after each external editing. Only then can you be sure that you have done your work correctly. I don’t know any editor who compiles the 4D source code or knows a corresponding lint.
And after the compilation process you can check in your work. That way you make life easier for your coworkers. And you’ll have the tokens in code anyway.

1 Like

I agree. Thinking about the things I actually write using the text editor it’s stuff like CSS, HTML, SVG, various JSON files, the odd text of info file. The only time I edit methods, for the great majority of time, is when I need to make project-wide changes. Like finding instances of a property name that has to be changed and may also be written in some HTML or support files. Or updating field names in an old database (one I work on has spaces in field names). The sort of thing where I’m using the find in files search capability that find in design can’t match.

As I understand it 4D is perfectly happy only looking at the tokens. The command names are just to make it human-readable. I just haven’t had an issue involving the tokens.

1 Like

The ideal workflow is not the real-world workflow. Our ERP consultants (not using 4D), regularly post source updates without compiling them first. Hell, in other tools I regularly post source updates without compiling them, first. They/I do that because the things they are doing are frequently small, or because they’re trying to get something out for everyone else to have a look at. So while I can say “Hey, muppet, compile your damn code before you submit a PR”, that is not a real-world workflow.
And it’s still harder to read
And it’s still harder to search for a particular line.
find me all the instances of
query ([employee];[employee].empno=

because once you tokenize that, if you don’t remember all of the tokens, the only way you’re searching for that code is in the ME, not your text editor.

From my understanding the idea of project mode structure as ‘text’ was to help with version control, not to make 4D methods editable in other text editors. 4D is sooo strongly typed that you should really use 4D IDE for all of the coding + why would you want to edit code in an editor that doesn’t allow debugging of the code vs the one that does ?
Tokenisation in exported methods can be a bit of a pain when it comes to ‘code review’ (say as a PR in github), it would be good if 4D had an option to export without tokenisation (even if it does take longer to parse, it’s 2020 though, not sure how text file parsing speed can be justification for tokenisation, especially only parsing files that have ‘changed’)

While at it, 4D could also recognize French and old commands.

If I insert something like “Au cas ou / Fin de cas” from a French forum, then 4D should convert it to “Case of / End case”.
Or vice versa, when a French developer inserts “Case of” (into a 4DM project file, or into the 4D IDE), then it should automatically turn to “Au cas ou”.
That should be no big deal, because there are only two languages: French and English.
The days when German developers had to write “Im Falle / eIm Falle” instead of “Case of / End case” are over. (This was really weird.)

And, if I insert “CALL PROCESS”, then 4D should convert it to “POST OUTSIDE CALL”.

Then we could get rid of the tokens.
And for those who love them, 4D could provide the option “Export 4DM project files with tokenization”.

2 Likes

What I’d like is a vision translator.
I think of something — 4D writes the code

3 Likes

I also looked into using different editors to work on 4D method files but finally came to the conclusion that it doesn’t work well. As stated in this thread it’s sometimes difficult to search, you don’t have access to the database structure and more.

The question is: why are there at least some people actively looking for ways to modify 4D code in different environments? I think that the answer is obvious: because the 4D environment is missing features. I would definitely love to have the power of Visual Studio to work with 4D code. But that only makes sense if I can easily edit files (without tokens), have full intellisense support, including the entire 4D programming language, all project methods, classes, forms etc. And of course debugging.

I don’t think that this really is something to expect. However, I definitely like to see enhancements to the experience a 4D developer has inside 4D. Would be cool to have 4D open a thread to collect feedback of most required features missing.

Here are some points from myself:

  • I cannot rearrange windows - would like to have the code on the left side, taking the entire space from the top to the bottom, except a tiny but rezisable area showing local variables, a watch window, call stack and so on. Yes, that is how it is being done in Visual Studio and it works very well. Compared to this, the 4D dev environment is static. I have a wide monitor (2560x1080), which is cool. So it would perfectly make sense to have some windows on the right to maximize the space for the actual code. But I can’t. 4D is using a lot of space at the top and I simply cannot change it.
  • 4D defines the way the code is formatted. I don’t want spaces before comments. I don’t want spaces behind method names. I like having theming support going way beyond what’s available.
  • The project explorer is a mess. There are tons of menu items to open it but really, it is a listbox. The home tab is useless as I cannot have folders with the same name (in different hierarchies). I cannot filter method names. I can rapidly type the first few characters to jump to a specific point, but often enough I’m hitting the wrong keys. A simple text box where I can enter meth@ to only see methods starting with meth would help.
  • And as a windows user I’m used to hit F2 to rename a method. I’m kind of a keyboard junkie and often feel that I’m slowed down by the environment. The computer should never be slower than I am. I feel that I have to use the mouse way to often.
  • I don’t have a tabbed UI in Windows. Instead 4D stores the window size and position of every method I had opened. When I open two instances of 4D to copy code from one instance to the other, I usually have both of them use 50% of the screen. If I previously moved a method window to the right, then I don’t see it in the smaller resolution.
  • and so on…

There are really good reasons why some of us are trying to use other editors. However, instead of trying to get everything into an external editor, we can also try to enhance the 4D environment. With “we” I mean 4D but actively listening to input from the community.

For the black theme: I created my own. I can post it if someone’s interested.
For methods without tokens: METHOD GET CODE delivers the code without tokens. Maybe that’s something to look at when searching for solutions to that question today.

1 Like

You me like the DWIM button?

(Do What I Mean)

Maybe, but in my case, I appreciate the convenience of opening a method without switching structures; I occasionally want to copy and paste between a dormant structure and my live application. For me, there is nothing missing in 4D and I have no problem using an external editor. I understand there are limits, but there are plenty of possibilities within those limits.

Yeah, it’s my personal opinion. People are different and developers are even more different. Guess that’s why tools like Visual Studio have so much user preferences. I’m well aware that 4D cannot build such a mighty environment. On the other hand, a little invest into the UI could potentially boost the developer’s performance. The more I like working with an environment, the more likely I will recommend it. Just my two cents.

Another nice feature on the Mac is to see the syntax colored code in the Finder preview and be able to look at it instantly with QuickLook. Unfortunately, there is something strange about .4dm files where I can’t get this to work unless I set the the file type to TEXT (e.g setfile -t “TEXT” _TEST4.4dm). 4DForm (as JSON) and 4DSettings (as XML) work as expected even though they are not set as TEXT files.

I’m using TextMate (free and open source) which has a built-in QuickLook plugin for any file type conforming to public.source-code.

The system maps file extensions (".4dm"), OS types (“TEXT”), UTIs (public.source-code) based on the information acquired from all installed apps, The mapping can be found in their Info.plist.

Technically, you could create a stub application (perhaps with Apple Script Editor) that declares the file extension “.4dm” as a relative of “public.source-code”.

Of course, you won’t get the syntax highlighting, you’d need to write a QuickLook plugin for that.

https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/understanding_utis/understand_utis_declare/understand_utis_declare.html

TextMate already has the QuickLook plugin which should work for any file designated as public.source-code. 4D 18 has that in the UTI. Part of the problem is that the system is not seeing it as that file type.

“mdls test.4dm” shows the info below.

kMDItemContentType                 = "dyn.ah62d4rv4ge8xk3dr"
kMDItemContentTypeTree             = (
   "dyn.ah62d4rv4ge8xk3dr",
   "public.item",
   "dyn.ah62d4rv4ge8xk3dr",
   "public.data"

I tried resetting the launch services database among a lot of other things. No luck. I’m wondering if the problem stems from having many 4D versions with different UTI information. 4dm looks like it first appeared in 4D 17 and it has different specification. public.plain-text instead of public.source-code.

I just watched the git presentation by Ahlam Benhachem, I feel we might be missing the big picture if we are simply looking at different editors and judging them for how capable (or not) there are at editing code. The primary reason for using external editors should be for version control.

In the presentation (around 20:00 in the English version), there is an example of using GitHub Desktop, VSCode and P4Merge to resolve conflicts. It would be a real pain to do this manually. In any situation, you won’t look at the git comments (stuff like <<<<<<< HEAD or @@ -13,11 +13,16 @@) directly. In the same way, I feel it doesn’t really matter if the code has tokens or not. You just let the desktop application visualise the conflicts and make an informed decision. This won’t happen for every diffs, only the diffs that can’t coexist requires intervention.

1 Like

@Keisuke_Miyako: Version control is just one of the many reasons to use an external editor. IMHO the main reason is that they have more features, settings, and options than the method editor, dare I say they are “better” as they are specialized tools that do that thing very well.
Getting git comments in a file is not necessarily a good thing, in fact, I find it annoying when it happens, because then you have to resolve the comment before you can recompile. If you have binary files in your repo, the git comments break the file.
Tokens are always going to reduce the utility that you get from being able to use an external editor.
Question: If a developer mass-substitutes in a project, does 4D go through the entire project when it opens it and retokenize it, or does the developer have to open every file that was modified so that it will be retokenized? I ask because in our ERP software, one of our consultants did a mass-substitution a couple of days ago, and updated 277 files.

Just checked it out:
4D does not retokenize all modified files.
Even opening a file by the developer (with 4D) will not retokenize it.
You need to modify the file with 4D to get it retokenized.