slowness

[Otto]

viewtopic.php?f=3&t=36635&p=218536&hilit=smb&sid=a166004cdfa2e421e0cdd92ae8e02696#p218536

[RAMESHBABU]

Hi Mr.Marco,

A small suggestion to you to try.

When the INDEXes are bigger and their travel time in NETWORK
environment, I am using the following technique(s)

1) USE <cDbfFile> ALIAS <cAlias> //(Don't open index file while using the DBF)
INDEX ON <cIndexKey> TAG <cIndexTag> MEMORY

The above command writes the INDEXes to the MEMORY of the Client System.
(Not on the server). So there would not be any load on the SERVER and there
would not be any travel time of INDEX Files from SERVER to CLIENT System.

Its working very well when I work with DBF Files in NETWORK environment.

2) I recommend the SERVER machine's OS is UBUNTU (not WINDOWS) with a
shared folder containing DATA. And provide the UNC path or a mapped drive,
to your client systems through an .INI file

I have been doing this past several years without any problem.

Please try this,

-Ramesh Babu P

[Otto]

Dear Ramesh,
Does that mean you no longer use index files at all?
I am working on a low-level adapter for DBF for PHP and am taking exactly this approach.
If that's the case, your experience with this is very interesting to me.
How large are your files at most when you create memory indexes?
Do you also happen to have any speed tests?
Do you happen to know where the source code for creating the memory files is?
Best regards,
Otto

[MarcoBoschi]

Dear Otto ,
I reported this because in my opinion there is a very very large bug somewhere
smb2, index files I don't know but it's absurd

> Marco, how many users are accessing simultaneously? How large are your databases?
only two users
11/09/2024 00:00 781.312 JALA.cdx
29/08/2024 13:33 6.525.890 Jala.dbf
29/08/2024 13:33 1.051.968 Jala.fpt

> It’s hard to give advice without knowing the specifics.

> I also noticed back then that when SMB-2 is enabled and you still have older Windows versions running on the network at the same time, it becomes > > >extremely slow.

We always disabled SMB-2 back then. Jimmy posted a good guide at that time.
> Why don’t you switch to Remote Desktop?

this is a very important question: if it were smb2 then why is it not slow even if I only open the dbf without opening the cdx?

[MarcoBoschi]

Dear Ramesh

Hi Mr.Marco,

>1) USE <cDbfFile> ALIAS <cAlias> //(Don't open index file while using the DBF)
>INDEX ON <cIndexKey> TAG <cIndexTag> MEMORY
I already do it
for heavy statistics I even copy everything locally
it's dramatically faster


>2) I recommend the SERVER machine's OS is UBUNTU (not WINDOWS) with a
>shared folder containing DATA. And provide the UNC path or a mapped drive,
>to your client systems through an .INI file
I I performed this test in a Synology nas it's veryu probabyl that it's based in linux

I have the same problem

[Otto]

Marco, What is the speed like when you work directly on the PC locally?
Best regards,
Otto

[MarcoBoschi]

Marco, What is the speed like when you work directly on the PC locally?

Otto the speed is similar
But If I open dbf and cdx in another session in the same pc it remains the same
I don'y have the slow down

I remember that many many years in a Netware access to a server volume was more fast than local hard sisk
using a 10 mbit lan coaxial cable!

[karinha]

Good morning Marcos. Is there a problem for you to make the files available:
09/11/2024 00:00 781.312 JALA.cdx
08/29/2024 13:33 6,525,890 Jala.dbf
08/29/2024 13:33 1,051,968 Jala.fpt
So I can create a new opening logic made with fivewin? For you to test on a network?

Buenos días Marcos. ¿Hay algún problema para que los archivos estén disponibles?
11/09/2024 00:00 781.312 JALA.cdx
29/08/2024 13:33 6.525.890 Jala.dbf
29/08/2024 13:33 1.051.968 Jala.fpt
¿Entonces puedo crear una nueva lógica de apertura hecha con fivewin? ¿Para probar en una red?

Gracias, tks.

Regards, saludos.

[MarcoBoschi]

Dear Karinha is not possible it contains confidential informations
I think this problem always occurs using any table dbfcdx

[karinha]

Dear Karinha is not possible it contains confidential informations
I think this problem always occurs using any table dbfcdx

No problem Marcos. It should be SLOW, just INDEXING.

No hay problema Marcos. Debería ser LENTO, solo INDEXAR.

Regards, saludos.

[RAMESHBABU]

Dear Mr.Otto,

Does that mean you no longer use index files at all?

I am using INDEXes. But not .CDX Files. I am creating INDEXes
in Local System's (Client System) Memory. So that, there should
not be any network delay, to set various index orders from Server.

How large are your files at most when you create memory indexes?

I have a students master file with 138 fields (Size 15 Mb) and having
indexes on 50 Fields. the Size of INDEX Files is almost 10 times bigger
than the DBF File.

This is the main file of my Application, which I use very frequently
through out my application. Every time I call it from client's system,
the speed was ok for 2 or 3 Users. After every additional user's connection
to it, used to create terrific delay. And my users used to complain above the speed.
Then I stopped creating physical indexes on the server and started creating
in local system.
[/quote]

Do you also happen to have any speed tests?

Yes. I made several time comparisons to create new indexes or adding
additional tags in server Vs creating indexes every time in local system.

My results were always moved me to adopt creation of temporary index on local
(client) machine.
[/quote]

[/quote]
Do you happen to know where the source code for creating the memory files is?

I am using xHarbour.

xHarbour Help manual says :
INDEX ON <cIndexKey> TAG <cIndexTag> TEMPORARY or MEMORY
OrdSetFocus(<cIndexTag>)

TEMPORARY / MEMORY (MEMORY CLAUSE IS NOT DOCUMENTED)
If this option is specified, a temporary index is created which is automatically destroyed when the index is closed.
The temporary index may be created in memory only or in a temporary file. This lies in the responsibility of the
RDD used for index creation.
[/quote]

The final point is Server OS. I tried to improve speed in several ways in WINDOWS Server OS (The hardware is not
real server). But I was quite un-happy with the speed. Then I tried to create a shared folder in UBUNTU server
and I found terrific speed, as if I was using my application in a local system.

These things are working for me and my clients are happy.

Best regards,

- Ramesh Babu P

[Otto]

Dear Ramesh,
thank you very much for your detailed response.
>I am using INDEXes. But not .CDX Files.
In other words, this means that you only have DBF files on your hard drive and no .CDX.



I do the same with my low-level functions. I read the records on the server and create a JSON, and at the same time, a JSON with the record numbers that match the filter conditions.

I have tested my functions against Enrico's EMG dBase editor in terms of speed - it is my reference when it comes to effective and fast work with DBF.

To create such an array with the record numbers, which corresponds to the index file, it takes about the same amount of time as with the EMG editor.

As soon as I have access to a LINUX server, I will also do a speed test. Currently, I only have Windows servers, unfortunately.

Best regards,
Otto

Best regards,
Otto

[Enrico Maria Giordano]

I have tested my functions against Enrico's EMG dBase editor in terms of speed - it is my reference when it comes to effective and fast work with DBF.

Thank you, Otto!

[Enrico Maria Giordano]

This is a reduced example of the problem. You have to open the same table on the server and then run this sample:

Code: Select all  Expand view  RUN

#include "Dbinfo.ch"


REQUEST DBFCDX
REQUEST DBFFPT


FUNCTION MAIN()

    LOCAL nSec

    RDDSETDEFAULT( "DBFCDX" )

    SET DBFLOCKSCHEME TO DB_DBFLOCK_CL53EXT

    USE \\yourserver\yourfolder\yourdbf SHARED

    SET INDEX TO \\yourserver\yourfolder\yourcdx

    nSec = SECONDS()

    WHILE !EOF()
        SKIP
    ENDDO

    ? SECONDS() - nSec

    INKEY( 0 )

    RETURN NIL

[Otto]

Enrico, not just my reference but also the first choice for daily work with DBF files.
Best regards,
Otto