Automated download of pictures

Re: Automated download of pictures

Postby Enrico Maria Giordano » Thu Mar 09, 2017 10:57 am

nageswaragunupudi wrote:Thank you.
GETURLTOFILE() is working with http: but not https:


This is not true. It would work if tipssl were linked (and opensll DLL's were available).

nageswaragunupudi wrote:URLDOWNLOADTOFILE() is working well with all and is also much faster than GETURLTOFILE().


I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?

nageswaragunupudi wrote:The actual reason for my testing is to find a faster alternative to the existing function WebPageContents(). Our requirement is to directly read the contents of the URL into memory.


I use this (but it needs opensll as well):

Code: Select all  Expand view
FUNCTION GETURL( cUrl )

    LOCAL cHtm := ""

    LOCAL oCli

    TRY
        oCli = TIPClientHttp():New( cUrl )

        IF EMPTY( oCli ) THEN BREAK

        oCli:nConnTimeout = -1

        IF !oCli:Open() THEN BREAK

        cHtm = oCli:ReadAll()

        IF EMPTY( oCli:cReply ) .OR. !( "OK" $ UPPER( oCli:cReply ) )
            cHtm = ""
        ENDIF
    CATCH
    END

    IF !EMPTY( oCli ) THEN oCli:Close()

    RETURN cHtm


EMG
User avatar
Enrico Maria Giordano
 
Posts: 8245
Joined: Thu Oct 06, 2005 8:17 pm
Location: Roma - Italia

Re: Automated download of pictures

Postby nageswaragunupudi » Thu Mar 09, 2017 1:25 pm

I was not aware the URLDOWNLOADTOFILE() is much faster than GETURLTOFILE(). Are you made any real test? Can you share the timing result?


I used the Tajmahal.jpg ( http not https )

GETURLTOFILE() --> 3.1 to 3.3 secs
Both
URLDOWNLOADTOFILE() and
MEMOWRIT( cFile, webpagecontents( curl ) ) --> 0.06 to 0.09 secs

Code: Select all  Expand view
function testweb4()

   local cURL := "http://cdn.history.com/sites/2/2015/04/hith-eiffel-tower-iStock_000016468972Large.jpg"
   local cFile := "test.jpg"
   local nSecs

   FERASE( cFile )
   URLDOWNLOADTOFILE( 0, cURL, cFile ) // Ignore time taken for the first time

   nSecs := SECONDS()
   GETURLTOFILE( cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 3.23

   nSecs := SECONDS()
   URLDOWNLOADTOFILE( 0, cURL, cFile )
   ? SECONDS() - nSecs, File( cFile ) // --> 0.07

   nSecs := SECONDS()
   MEMOWRIT( cFile, webpagecontents( curl ) )
   ? SECONDS() - nSecs, File( cFile )  // --> 0.07

   if File( cFile )
      XIMAGE( cFile )
   endif

return nil
Regards

G. N. Rao.
Hyderabad, India
User avatar
nageswaragunupudi
 
Posts: 10208
Joined: Sun Nov 19, 2006 5:22 am
Location: India

Re: Automated download of pictures

Postby Enrico Maria Giordano » Thu Mar 09, 2017 2:46 pm

That's only a cache effect. Try calling this before download:

Code: Select all  Expand view
DLL FUNCTION DELETEURLCACHEENTRY( cUrl AS LPSTR ) AS BOOL;
    PASCAL FROM "DeleteUrlCacheEntryA" LIB "wininet.dll"


EMG
User avatar
Enrico Maria Giordano
 
Posts: 8245
Joined: Thu Oct 06, 2005 8:17 pm
Location: Roma - Italia

Re: Automated download of pictures

Postby nageswaragunupudi » Fri Mar 10, 2017 5:32 pm

Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available
Regards

G. N. Rao.
Hyderabad, India
User avatar
nageswaragunupudi
 
Posts: 10208
Joined: Sun Nov 19, 2006 5:22 am
Location: India

Re: Automated download of pictures

Postby Enrico Maria Giordano » Fri Mar 10, 2017 6:09 pm

nageswaragunupudi wrote:Mr EMG

You are right.
Without the benefit of cache, GETURLTOFILE() is faster than the other two.
Another observation is that GETURLTOFILE() does not take advantage of cache and other two take advantage of cache if available


That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

EMG
User avatar
Enrico Maria Giordano
 
Posts: 8245
Joined: Thu Oct 06, 2005 8:17 pm
Location: Roma - Italia

Re: Automated download of pictures

Postby nageswaragunupudi » Sat Mar 11, 2017 3:27 am

That's true. But use of cache means that the file is not really downloaded, even if it's changed. So use it with caution.

Yes. Thanks
Regards

G. N. Rao.
Hyderabad, India
User avatar
nageswaragunupudi
 
Posts: 10208
Joined: Sun Nov 19, 2006 5:22 am
Location: India

Previous

Return to FiveWin for Harbour/xHarbour

Who is online

Users browsing this forum: MarcoBoschi and 10 guests