access to a supplier web site under program control

[Rick Lipkin]

To All

I have a request from a Customer who is a small engine parts dealer that routinely logs in to multiple suppliers web sites and checks availability of specific part numbers. He has asked if it is possible to create a program that will access and login to multiple suppliers web sites in one program and retrieve parts availibility, his cost information and display a grid of each suppliers information so he can choose the best price.

In reviewing the requirements and looking at various Vendor web sites it appears that each supplier has a specific login, userid to access their parts information. What I know I will need is login credentials and the vendors IP address for each site .. with those three variables, how would I connect to a suppliers site under (x)Harbour\FWH control ( sockets perhaps ) ?

I realize this is a vague description of the process, and I will probably have to look at their web page to track the html?? .. I was just curious if anyone has actually remotely logged into a website, passing login parameters and making a connection using (x)Harbor and FWH ?

Would be grateful for any suggestions.

Thanks
Rick Lipkin

[lailton.webmaster]

Search on google by: curllib
make download of curllib for DOS and look samples inside of file downloaded. using he you can execute operation on website request values etc...

Maybe it can help you.

[James Bott]

Rick,

Try searching this forum for "login." This thread is particularly interesting:

viewtopic.php?f=3&t=12514&start=0&hilit=https+patrick

I have not tried any of this so I can't offer any specific help.

Regards,
James

[Rick Lipkin]

James

Thank you for your help .. I see there is a userid and login test code .. I will have to research this code further.

I have a meeting this week with our client and will hopefully get some examples of his vendor web sites.

Appreciate your help !

Rick

[Rick Lipkin]

To All

Reviewing the topic James pointed out .. it has led me to this simple active x code :

#include "fivewin.ch"

Function Main()
Local oWnd,oActiveX

DEFINE WINDOW oWnd TITLE "Test"

   oActiveX = TActiveX():New( oWnd, "Shell.Explorer" )

   oWnd:oClient := oActiveX // To fill the entire window surface

   oActiveX:Do( "Navigate2", "https://www.dixiesales.com/login-us.aspx" )

ACTIVATE WINDOW oWnd MAXIMIZED

return(nil)

The above code takes me to where I want to be but I need to be able to pass a dealer number to the login as a parameter.

Eventually I want to be able to 'crawl' multiple vendor sites, login and pass a stock number and get availability and dealor cost.

Any ideas would be appreciated.

Thanks
Rick Lipkin

[James Bott]

Rick,

I think you need something like the code below. I could not get it working with my version of FWH (I think it is 10.06). The activeX object never gets instantiated properly.

You will need to change the 1000 to whatever Dealer number you need.

James

Code: Select all  Expand view

/*
Purpose: Test auto login to website
*/


#include "fivewin.ch"

function Main()

   local oWnd, oActiveX

msgInfo() // this shows

   DEFINE WINDOW oWnd

   @ 0, 0 ACTIVEX oActiveX PROGID "Shell.Explorer" OF oWnd

msgInfo( valtype( oActiveX ) )  // This never shows

   oWnd:oClient = oActiveX


msginfo()

   ACTIVATE WINDOW oWnd //;
      ON INIT oActiveX:Do( "Navigate2",;
                           "https://www.dixiesales.com/login-us.aspx",,,;
                           GetPostData( "Dealer Number=1000" ),;
                           "Content-Type: application/x-www-form-urlencoded" + CRLF )

return nil

//-----------------------------------------------------------------------------------------//

#pragma BEGINDUMP

#include <hbapi.h>
#include <windows.h>

HRESULT hb_oleVariantToItem( PHB_ITEM pItem, VARIANT * pVariant );

HB_FUNC( GETPOSTDATA )
{
   VARIANT vPostData = {0};
   LPSAFEARRAY psa;
   LPCTSTR cszPostData = hb_parc( 1 );
   UINT cElems = lstrlen( cszPostData );
   LPSTR pPostData;

   VariantInit( &vPostData );

   psa = SafeArrayCreateVector( VT_UI1, 0, cElems );
   if( ! psa )
   {
      hb_retnl( E_OUTOFMEMORY );
      return;
   }

   SafeArrayAccessData( psa, ( LPVOID * ) &pPostData );
   memcpy( pPostData, cszPostData, cElems );
   SafeArrayUnaccessData( psa );

   V_VT( &vPostData ) = VT_ARRAY | VT_UI1;
   V_ARRAY( &vPostData ) = psa;

   hb_oleVariantToItem( hb_param( -1, HB_IT_ANY ), &vPostData );
}

#pragma ENDDUMP

//end

[Rick Lipkin]

James

I think this is more complicated that it appears ... I got your code modifications to run .. but I have no idea what the field name is for the login .. I looked at the html code from the page and I need to try this code on a site I have some login credentials on ..

Thank you !

Rick

//---------------------
#include "FiveWin.ch"

function Main()

local oWnd, oActiveX

DEFINE WINDOW oWnd

@ 0, 0 ACTIVEX oActiveX PROGID "Shell.Explorer" OF oWnd

oWnd:oClient = oActiveX

ACTIVATE WINDOW oWnd ;
ON INIT oActiveX:Do( "Navigate2",;
"https://www.dixiesales.com/login-us.aspx",,,;
GetPostData( "name=1000" ),;
"Content-Type: application/x-www-form-urlencoded" + CRLF )
return nil

//---------------------------------------------------------------------
#pragma BEGINDUMP

#include <hbapi.h>
#include <windows.h>

HRESULT hb_oleVariantToItem( PHB_ITEM pItem, VARIANT * pVariant );

HB_FUNC( GETPOSTDATA )
{
VARIANT vPostData = {0};
LPSAFEARRAY psa;
LPCTSTR cszPostData = hb_parc( 1 );
UINT cElems = lstrlen( cszPostData );
LPSTR pPostData;

VariantInit( &vPostData );

psa = SafeArrayCreateVector( VT_UI1, 0, cElems );
if( ! psa )
{
hb_retnl( E_OUTOFMEMORY );
return;
}

SafeArrayAccessData( psa, ( LPVOID * ) &pPostData );
memcpy( pPostData, cszPostData, cElems );
SafeArrayUnaccessData( psa );

V_VT( &vPostData ) = VT_ARRAY | VT_UI1;
V_ARRAY( &vPostData ) = psa;

hb_oleVariantToItem( hb_param( -1, HB_IT_ANY ), &vPostData );
}

#pragma ENDDUMP

[Rick Lipkin]

Laiton

I did some research into 'curl' ... and by their own definition:

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP,
FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP,
SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.The 'curl' instructions look interesting but I do not know if this program can be used to just open a url and pass parameters to 'login' to the site ?

Do you have an example you might share with me using 'curl' that I can test .. I do not want to send files ( ftp ..etc ) I just want to be able to login to a site under program control.

Thanks
RIck

[lailton.webmaster]

Rick,

http://www.fiveweb.com.br/curl-test/curl-test.rar

Note I access this url with Curl.exe make login and return me result:
http://www.fiveweb.com.br/curl-test/


( I don't know if is it that you wanna, more please test. )

[Rick Lipkin]

Lailton

WOW .. I ran your demo and looked at your code .. this certainly logs into your site .. I will do some more testing with your example ..

I appreciate your help and quick response !

Rick

[James Bott]

Rick,

After rereading your original post, it seems that logging in is only the tip of the iceberg. What you really need is for the website to provide you with a XML file containing the data you need. I believe this was originally done with SOAP which Microsoft has since abandoned, but there must be a replacement technology. This probably is worth some Googling. You might also search this forum for SOAP and XML.

Regards,
James

[lailton.webmaster]

Good Rick,

If you wanna to get xml file really you can use soap kit of microsoft ( for webservice ), work fine.

more if is only to login you can use curllib.

Good Luck

[Rick Lipkin]

Laiton and James ..

Definitly the tip of the iceburg .. and I will need to get with one of the vendors to see how they query up a stock number so I can tap into that process ..

Really appreciate the advice you both have shared !

Rick

[Rick Lipkin]

Lailton

What is the trigger ( submit ) input to get to the next page ? I looked at your Receive.html and reviewed the source ..

Your code evals to the connect logic and returns the html of the page .. but I do not exactically know what 'curl' is doing .. or how to get past a login page once the connection is successful ?

Hope that made sense ??

Thanks
Rick

[TimStone]

Rich,

You are trying to re-create the wheel ! Most parts suppliers already use software that makes the data available to their clients ... but there is a monthly access fee from the "source" company. They probably can't let you access the data because of the licensing agreement they have with the vendor of that product. You can program to it, but your upfront cost to be "licensed" will be high.

Before the client sets you on a path that costs you a lot, you may want to investigate this more closely with the vendors.

Tim