Collecting all samples (source) from the forum

[Marc Venken]

Hello,

I'v spend more time i looking up samples, topics enz... than actualy programming. (I don't mind, since it's only for personel use) and the learing curve is nice.

Maybe it is possible to write a program that is able to read all topics, and when the topic contains the strings [code][/code], it could take
the

Subject, Date, Auther, forum (en,Spa) and the link like : posting.php?mode=post&f=3

When browsing this program, we could locate more topics with source code included samples.

In my case, all I do is take samples, and rebuild, alther them.

The search function of forums can't do this, but FWH CAN !

Just a idea...

[ukoenig]

Marc,

what I can do is, creating a downloadlist of all my tools with infos.
Over the years there are many, scattered inside the forum.

regards
Uwe

[Marc Venken]

Uwe,

That would be great !

I've already found some nice samples from you They are with all files included and so very interesting for me. Probably also for some others I hope...

The samples in FWH are also nice, but some are very old. Recent stuff on the Forum is better.

All samples of Mr. Rao are interesting, since XBrowse is my engine in my program. Maybe they are also located on a folder on his computer?

Thanks for the effort.

Marc

[nageswaragunupudi]

Hello,

I'v spend more time i looking up samples, topics enz... than actualy programming. (I don't mind, since it's only for personel use) and the learing curve is nice.

Maybe it is possible to write a program that is able to read all topics, and when the topic contains the strings

Code: Select all | Expand

, it could take
the

Subject, Date, Auther, forum (en,Spa) and the link like : posting.php?mode=post&f=3

When browsing this program, we could locate more topics with source code included samples.

In my case, all I do is take samples, and rebuild, alther them.

The search function of forums can't do this, but FWH CAN !

Just a idea...

Please try this sample. You can improve on this:

Code: Select all | Expand

#include "fivewin.ch"

REQUEST DBFCDX

static nLastTopic := 33507  // rage of topics
static nFirstopic := 33400

//----------------------------------------------------------------------------//

function Main()

   SET DATE BRITISH
   SET CENTURY ON
   SET DELETED ON
   RDDSETDEFAULT( "DBFCDX" )

   DBCREATE( "SAMPLES.DBF", { ;
      { "TOPICNO",   'N',  6, 0 }, ;
      { "TOPIC",     'C', 60, 0 }, ;
      { "AUTHOR",    'C', 40, 0 }, ;
      { "DATE",      'D',  8, 0 }, ;
      { "CODE",      'M', 10, 0 }  }, ;
      "DBFCDX", .T., "DB" )
   FW_CdxCreate()
   CLOSE DB

   USE SAMPLES EXCLUSIVE VIA "DBFCDX"

   ForumSamples( nLastTopic, nFirsTopic )

   BrowseSamples()

return nil

//----------------------------------------------------------------------------//

function BrowseSamples()

   local oDlg, oFont, oBold, oMono, oGet, oBrw

   SET ORDER TO TAG TOPICNO
   GO TOP

   DEFINE FONT oFont NAME "Segoe UI" SIZE 0,-14
   DEFINE FONT oBold NAME "TAHOMA" SIZE 0,-18 BOLD
   DEFINE FONT oMono NAME "Lucida Console" SIZE 0,-12

   DEFINE DIALOG oDlg SIZE 900,700 PIXEL TRUEPIXEL FONT oFont ;
      TITLE "SAMPLES IN FWH FORUMS"

   @  90,20 XBROWSE oBrw SIZE 400,-20 PIXEL OF oDlg ;
      DATASOURCE "SAMPLES" ;
      COLUMNS "TOPICNO", "DATE", "AUTHOR" ;
      AUTOSORT ;
      LINES NOBORDER

   WITH OBJECT oBrw
      :nMarqueeStyle := MARQSTYLE_HIGHLROWRC
      :bChange       := { || oDlg:Update() }
      :lIncrFilter   := .t.
      :bSeek         := { |c| ( oBrw:cAlias )->( BrwFilter( c ) ) }
      :CreateFromCode()
   END

   @ 20, 20 SAY TRIM( SAMPLES->TOPIC ) SIZE 860,30 PIXEL OF oDlg CENTER ;
         FONT oBold UPDATE

   @ 60, 20 SAY "Filter containing all words any where" SIZE 300,20 PIXEL OF oDlg

   @ 60,340 SAY oBrw:oSeek PROMPT oBrw:cSeek SIZE 540,20 PIXEL OF oDlg ;
      COLOR CLR_HRED,CLR_YELLOW

   @  90,420 SAY "CODE" SIZE 460,30 PIXEL OF oDlg CENTER ;
      COLOR CLR_BLACK, nRGB( 231, 242, 255 )

   @ 120,420 GET oGet VAR SAMPLES->CODE SIZE 460,540 PIXEL OF oDlg ;
      MEMO READONLY FONT oMono UPDATE

   oDlg:bPainted := { || oDlg:Box( 59,339,81,881 ) }

   ACTIVATE DIALOG oDlg CENTERED
   RELEASE FONT oFont, oMono, oBold

return nil

//----------------------------------------------------------------------------//

function BrwFilter( c )

   local lFound   := .t.
   local aTokens
   local cSaveFilter := DBFILTER()
   local nSaveRec    := RECNO()
   local cFilter     := {}

   if Empty( c )
      return .t.
   endif

   c  := UPPER( c )
   aTokens  := HB_ATokens( c )

   for each c in aTokens
      AAdd( cFilter, "'" + c + "' $ UPPER( DBRECORDINFO( 9 ) )" )
   next

   cFilter  := FW_ArrayAsList( cFilter, " .AND. " )

   SET FILTER TO &cFilter
   GO TOP
   lFound   := ( OrdKeyCount() > 0 )

return lFound

//----------------------------------------------------------------------------//

function ForumSamples( nTopic, nLast )

   local cTopic, cUrl, cPageURL, cUser, cText, cCode, nPage, nPages, n, cLeft, dDate

   DEFAULT nTopic   := 33507, nLast := nTopic - 50

   for nTopic := nTopic to nLast step -1

      nPage    := 1
      cUrl  := TopicNoToURL( nTopic )
      do while .t.
         cPageURL := cUrl + If( nPage > 1, "&start=" + LTrim( Str( nPage * 15 ) ), "" )
         MsgRun( cPageURL, "READING FORUM PAGE", { || ;
            cText := WebPageContents( cPageUrl, .t. ) ;
            } )

         if nPage == 1
            nPages   := PageCount( cText )
            cTopic   := textbetween( ctext, "<h2>", "</h2>", 1 )
            cTopic   := textbetween( cTopic, ">", "</a>", 1 )
         endif

         n     := 1
         do while !Empty( cCode := TextBetween( cText, "<code>", "</code>", n, @cLeft ) )
            cUser := GetUserName( cLeft, @dDate )
            if Empty( dDate )
               dDate := CTOD( "" )
            endif
            cCode          := ExtractPrgCode( cCode )
            //
            DBAPPEND()
            FIELD->TOPICNO := nTopic
            FIELD->TOPIC   := cTopic
            FIELD->AUTHOR  := cUser
            FIELD->DATE    := dDate
            FIELD->CODE    := cCode
            n++
         enddo
         nPage++
         if nPage > nPages
            EXIT
         endif
      enddo
   next nTopic

return nil

//----------------------------------------------------------------------------//

function TopicNoToURL( nTopic )
return   "http://forums.fivetechsupport.com/viewtopic.php?f=3&t=" + cValToChar( nTopic )

//----------------------------------------------------------------------------//

function TextBetween( cText, cStartTag, cCloseTag, nPos, cLeft, cRight )

   local cRet  := ""

   if !( cStartTag $ cText )
      cLeft    := cText
      cRight   := ""
      return ""
   endif

   cRight   := AfterAtNum( cStartTag, cText,  nPos )
   cRet     := BeforAtNum( cCloseTag, cRight, 1    )

   if PCount() > 4
      cLeft    := BeforAtNum( cStartTag, cText,  nPos )
      cRight   := AfterAtNum( cCloseTag, cRight, 1    )
   endif

return cRet

//----------------------------------------------------------------------------//

function ExtractPrgCode( cCode )

   local nFrom, nUpto, cLeft, cRight, cToken
   local nFor
   local aSubs := { ;
      { '<br />',CRLF }, ;
      { '&nbsp;'," " }, ;
      { 'ÿ'," " }, ;
      { '&quot;','"' } }

   for nFor := 1 to Len( aSubs )
      cCode    := StrTran( cCode, aSubs[ nFor, 1 ], aSubs[ nFor, 2 ] )
   next

   do while !Empty( cToken := TextBetween( cCode, "<", ">", 1, @cLeft, @cRight ) )
      cCode    := cLeft + cRight
   enddo

   aSubs := { ;
      { '&gt;', ">"  }, ;
      { '&lt;', "<"  } }

   for nFor := 1 to Len( aSubs )
      cCode    := StrTran( cCode, aSubs[ nFor, 1 ], aSubs[ nFor, 2 ] )
   next
   do while !Empty( cToken := TextBetween( cCode, "&#", ";", 1, @cLeft, @cRight ) )
      cToken   := Chr( Val( cToken ) )
      cCode    := cLeft + cToken + cRight

   enddo

return cCode

//----------------------------------------------------------------------------//

function PageCount( cText )

   local nAt
   local nPages   := 1

   if ( nAt := AT( "Page <strong>", cText ) ) > 0
      cText    := SubStr( cText, nAt + 14, 50 )
      nPages   := Val( AfterAtNum( "<strong>", cText, 1 ) )
   endif

return nPages

//----------------------------------------------------------------------------//

function GetUserName( cText, dDate )

   local c1    := "/memberlist.php?mode=viewprofile&amp;u=" //2342">cnavarro</a></strong> &raquo; Tue Jan 17
   local c2    := ["username]
   local nAt   := RAT( c1, cText )
   local n2    := RAT( c2, cText )
   local cUser := ""
   local cDate

   nAt      := Max( nAt, n2 )
   if nAt > 0
      cText    := SubStr( cText, nAt, 200 )
      cUser    := TextBetween( cText, ">", "<", 1 )
      cDate    := AllTrim( TextBetween( cText, "&raquo;", "</p>" ) )
      cDate    := Upper( AfterAtNum( " ", cDate, 1 ) )
      dDate    := uCharToVal( cDate, 'D' )
   endif

return cUser

//----------------------------------------------------------------------------//

[Marc Venken]

AWESOME !!

This is realy a great example and a very usefull tool !

Maybe we can make a project out of this and put it on top of the forum as a locked item.

I'm sure that many people could benifit from this source and project.

We could Mark Items, and put them in a category that we create for ourselve as interesting.
I can think of many more.

Anybody interested in making this a genereal forum tool ?

[ukoenig]

It is very interesting

I think adding the headline ( topic title ) and a maybe provided download-link would be useful.
Because of big sourcefiles, frequently I''m including a download-link with the complete sample or tool.

regardss
Uwe

[bpd2000]

Excellent , Thank you Mr. Rao

[Antonio Linares]

I would add the topic title and also the possibility to save it to a DBF so next time it will start faster

[ukoenig]

Is it possible to include download-links as well and execute the download,
the same like open them from inside a topic ?



regards
Uwe

[Marc Venken]

This sample is calling the FW board for every topic in a loop, and is then downloading the source.

Could It be possible with this code to download pic's in a loop from a server ?
Pics are maybe not possible

I've a file with these items

https://cdn.sportdirect.com/resizer/500 ... 000-01.jpg
https://cdn.sportdirect.com/resizer/500 ... 000-01.jpg
https://cdn.sportdirect.com/resizer/500 ... 000-01.jpg
https://cdn.sportdirect.com/resizer/500 ... 040-01.jpg
https://cdn.sportdirect.com/resizer/500 ... 040-01.jpg

I would like to retrieve the pics localy.