Fulltextsearch (to Marco Boschi)

[Otto]

Hello Marco,
did you ever found a solution to your question.
Thanks in advance
Otto

#include "FileIO.ch"

FUNCTION MAIN()

LOCAL cAllMemos

// open and read all dbf file into cStringa variable
nHandle := FOPEN( "memo2.fpt" )
nSize := FSeek( nHandle, 0, FS_END )
FSeek( nHandle, -nSize, FS_RELATIVE )
cAllMemos := SPACE( nSize )
nRead := FRead( nHandle, @cAllMemos, nSize )
FCLOSE( nHandle )
? cAllMemos
RETURN NIL

cAllMemos contains in a single fRead all fpt file.

For instance I search string "MARCO" in cAllMemos string.
Obviously I know structure of MEMO2.DBF

The question is: is it possible to get the number of record and the
name of memo field that corresponds to this search?

what are the rules for storing contents of memo fields in the file
fpt?
Does exist some any documentation?




@Harbour
FlexFile3


*In this first memo field I write "MARCO"
¯
In this I write
"BOSCHI" ¯

[MarcoBoschi]

Otto,
take a look at
http://groups.google.com/group/comp.lang.xharbour/browse_thread/thread/c48a5aff0123c410/fef17ea5e6d08c8d?lnk=gst&q=allmemo#fef17ea5e6d08c8d

Sorry but I'm very busy today and tomorrow

For the moment, I tell you only one thing:
for fast searches with good results I'm trying to load an enture table into an array and then search the array.
Bye

[James Bott]

Here is documentation for FTP memo fields:

http://www.clicketyclick.dk/databases/x ... FPT_STRUCT

Regards,
James

[Otto]

Hello James,
thank you for your help.
If I use a text editor to look I have a value of 73610 for example inside the memo field.
Do you know what this value is. I think it is the offset of the memoblock. But how do I calculate?
I also saw that if a memoblock is edited and larger than 512 kB a new block is added. The old one still
remains and I didn’t find a delete mark.
As Marco said it seems impossible with the oneway design of memofields - the offset is only stored in the dbf file -
to find the record number corresponding to the memoblock.
What do you think?


Best regards,
Otto

[James Bott]

Otto,

Sorry, I have never done any work requiring working with the structure of memo fields, I just happen to have the link to the reference in my notes.

Perhaps you can try Googling for "FTP memo field structure reading" or some such.

Regards,
James

[Otto]

Hello James,
I spend my last weekend Googling for "FTP memo field structure" with no success.
Best regards,
Otto

[James Bott]

Otto,

Well, so much for that idea!

Can you tell us more about what you are trying to do? I assume you need to search memo fields, but in how big of a file? Also, it is a one time search or multiple searches at the same time (by same time I don't mean all at once, but sequential searches all by the same user).

One of the difficulties, as you have mentioned, is unused blocks of text. Memo fields leave lots of unused blocks of text in the file which causes the file size to grow very quickly. To be efficient you have to either skip over those blocks or pack the file regularly.

I vaguely remember that there was a third-party commercial product that used a different file structure to store memo fields. I also remember working on one myself a number of years ago. Since you are using database objects it should be fairly simple to substitute a different database class (that doesn't use memo fields) into your system without any code changes to your system (only new code in the database class).

Anyway, if you tell me more about your problem, maybe I can come up with some ideas.

Regards,
James

[Gale FORd]

I think you can any text ( including memo fields ) in a FTS (Fast Text Search) or Hyper-SEEK index. It will take all words in the text/memo field and index them.

[James Bott]

Gale,

If I remember correctly, those are only available with the commercial version of xHarbour. Is that right?

Regards,
James

[StefanHaupt]

James,

HiPer-SEEK is available in free xharbour too. You can search a string over several fields.

Here is a sample (from xharbour)

#include "dbinfo.ch"

//REQUEST RMDBFCDX

PROCEDURE Main()
FIELD FIRST, LAST, STREET, CITY
LOCAL n, hs

if ascan( rddList(1), "RMDBFCDX" ) != 0
rddSetDefault( "RMDBFCDX" )
endif

use test
hs := HS_INDEX( "test", "FIRST+LAST+STREET+CITY", 2, 0, , .T., 3 )

/* Look for all records which have 'SHERMAN' string inside */
HS_SET( hs, "SHERMAN" )
while ( n := HS_NEXT( hs ) ) > 0
dbgoto( n )
if HS_VERIFY( hs ) > 0
? rtrim( FIRST+LAST+STREET+CITY )
endif
enddo
wait

/* Does RDD support Record Map Filters? */
if dbinfo( DBI_RM_SUPPORTED )
/* if yest then let set filter for all records with 'SHERMAN'
word and look at them in browser */
HS_FILTER( hs, "SHERMAN" )
dbgotop()
browse()
endif
HS_CLOSE( hs )
RETURN

[nageswaragunupudi]

Is RMDBFCDX available in free xHarbour? If so, what is the library to be linked to use RMDBFCDX ?

[StefanHaupt]

RMDBFCDX is only available in commercial xharbour, but you don´t need it to run the sample. Only if it is linked in, it´s used. You can also comment out these lines:

if ascan( rddList(1), "RMDBFCDX" ) != 0
rddSetDefault( "RMDBFCDX" )
endif

[nageswaragunupudi]

I am getting HS_KEY, HS_FILTER as unresolved externals. What is/are the libaray(ies) I need to link to include these functions?

[Gale FORd]

I believe it is only in the commercial version.

[James Bott]

Stefan,

HiPer-SEEK is available in free xharbour too. You can search a string over several fields.

Does this include memo fields?

Even so, I would think an index that indexes each word in a memo field would be much more efficient. This would be like a many-to-many relationship. Each word could be listed multiple times once for each record containing it.

Regards,
James