Question about Memo Field Storage Strategy?

[Otto]

Hello friends,

The tests with low-level file access to DBF are encouraging. One can forget about so much overhead - please note that I am making these considerations for use on a WEBSERVER or in an RDP environment.
Also, the file sizes in the area I am working in are manageable.

I also think that for your own application, you do not necessarily have to follow the standard.

I have a question about the storage of memo fields and their impact on performance, especially assuming that you are using the current version of Windows and have a fast SSD installed.

Specifically, I am considering whether it makes sense to store memo fields in separate files.

Does anyone have experience or recommendations on this topic under Windows with a fast SSD?
Are there any best practices or special tools that are particularly suitable for such scenarios? I look forward to your opinions and tips!

Thank you in advance!
Best regards,
Otto

[nageswaragunupudi]

Theoretically, keeping separate files improves performance.
But we perceive difference in performance only in case of
1. Large files
2. Access DBFs over network ( LAN or WAN )
In other cases, keeping memos in the same table simplifies programming.

[cnavarro]

Dear Otto, remember that Harbour does not allow encrypting tables with memo fields.

[Otto]

Dear Christobal,

Thank you.

I have different requirements compared to many here; my databases contain few records, usually no more than 30,000-100,000 records.
I also have no network access, as I work on the server via RDP or with mod_harbour.

What is significantly increasing are notes about customers or products, which I have previously inserted into the memo file. Also, pictures.
Having these "memos" in different files allows me to start a full-text search with PS and have immediate direct access to the hits:
Link to forum post
viewtopic.php?f=3&t=44596&p=270763&hilit=powershell&sid=f305aab7597289c7306a74f445bb62cf#p270763

>This simplifies programming.
I am currently working on low-level access to DBF.
I think that once I have implemented the basic functions, it will be very easy and extremely fast.
You don't even notice how much unnecessary overhead you accumulate with RDD.

Indexes are still a problem, but I think that with fast SSDs, indexes don't affect filtering and access that much. So, I plan to dismiss them.

SQL is not an alternative for such lightweight, small applications like mine. Additionally, you are limited in developing ideas with fixed formats.

When I worked with "FiveWin for Pocket PC," we didn't have a database either. Back then, I used lock files similar to how Excel or Word does it.
I think now with mod_harbour, I will solve it similarly. rlock() and flock() seem a bit unstable on the web.
It is also dangerous if the indexes are not 100% synchronized because a wrong record might be updated or deleted.

I think that the file system is underutilized as a database structure. It is difficult for us older folks to rethink.
I can still remember how the speed would drop with Windows 2000 and earlier if you had many files in a directory, and I recall that 20,000 was a limit.

But today, everything is fast and without limits.

Best regards,

Otto