Artificial intelligence - Class TPerceptron

Artificial intelligence - Class TPerceptron

Postby Antonio Linares » Wed Apr 19, 2017 8:34 am

This is a simple example to start learning the very basics of the Artificial Intelligence

In this example we create a TPerceptron object and we make it learn the behavior of the function nAnd(), then we check the results and they seem to be right.
Then we create a new TPerceptron object and we make it learn the behavior of the function nOr(), the results seem right again :-)

I am not an expert at all on IA, I am just curious about it. So I appreciate your comments and tests

perceptron.prg
Code: Select all  Expand view
#include "FiveWin.ch"

function Main()

   local oPerceptron := TPerceptron():New( 2 )
   local n, nX, nY
   
   // the perceptron learns nAnd()
   for n = 1 to 1000
      oPerceptron:Learn( { nX := nRandom( 1 ), nY := nRandom( 1 ) }, nAnd( nX, nY ) )
   next
   
   MsgInfo( oPerceptron:CalculateOutput( { 1, 1 } ), "1" )  
   MsgInfo( oPerceptron:CalculateOutput( { 1, 0 } ), "0" )
   MsgInfo( oPerceptron:CalculateOutput( { 0, 1 } ), "0" )
   MsgInfo( oPerceptron:CalculateOutput( { 0, 0 } ), "0" )

   // We create a new perceptron to learn nOr()
   oPerceptron := TPerceptron():New( 2 )

   for n = 1 to 1000
      oPerceptron:Learn( { nX := nRandom( 1 ), nY := nRandom( 1 ) }, nOr( nX, nY ) )
   next
   
   MsgInfo( oPerceptron:CalculateOutput( { 1, 1 } ), "1" )
   MsgInfo( oPerceptron:CalculateOutput( { 1, 0 } ), "1" )
   MsgInfo( oPerceptron:CalculateOutput( { 0, 1 } ), "1" )
   MsgInfo( oPerceptron:CalculateOutput( { 0, 0 } ), "0" )
   
   XBrowser( oPerceptron )

return nil

CLASS TPerceptron

   DATA aPreviousWeights
   DATA nPreviousThreshold
   DATA aWeights          
   DATA nThreshold        
   DATA nLearningRate    
   
   METHOD New( nInputs )
   
   METHOD Learn( aInputs, nExpectedOutput )
   
   METHOD CalculateOutput( aInputs )

ENDCLASS

METHOD New( nInputs ) CLASS TPerceptron

   local n

   ::nPreviousThreshold = 0
   ::aWeights = Array( nInputs )
   ::aPreviousWeights = Array( nInputs )
   ::nThreshold = 0
   ::nLearningRate = 0.1

   for n = 1 to nInputs
      ::aPreviousWeights[ n ] = 0
      ::aWeights[ n ] = 0
   next  

return Self

METHOD Learn( aInputs, nExpectedOutput ) CLASS TPerceptron

   local nError := nExpectedOutput - ::CalculateOutput( aInputs )
   local n
   
   for n = 1 to Len( ::aWeights )
      ::aWeights[ n ] = ::aPreviousWeights[ n ] + ::nLearningRate * nError * aInputs[ n ]
   next
   
   ::nThreshold = ::nPreviousThreshold + ::nLearningRate * nError

   ::aPreviousWeights = ::aWeights
   ::nPreviousThreshold = ::nThreshold

return nil

METHOD CalculateOutput( aInputs ) CLASS TPerceptron

   local nSum := 0
   local n
   
   for n = 1 to Len( ::aWeights )
      nSum += aInputs[ n ] * ::aWeights[ n ]
   next  
       
   nSum += ::nThreshold

return If( nSum > 0, 1, 0 )
regards, saludos

Antonio Linares
www.fivetechsoft.com
User avatar
Antonio Linares
Site Admin
 
Posts: 41205
Joined: Thu Oct 06, 2005 5:47 pm
Location: Spain

Re: Artificial intelligence - Class TPerceptron

Postby Antonio Linares » Wed Apr 19, 2017 10:14 am

It is a very simple concept:

A perceptron it is equivalent to a brain neuron.

The perceptron receives "inputs" and each of these have more or less intensity ("weights"), and the perceptron acts as a Harbour function (method "CalculateOutputs") receiveing some parameters (the inputs) and returning a result to know if it is or not the expected result ("nExpectedOutput")

The perceptron modifies its weights (DATA aWeights) if the result is not ok, so it "adapts" to the result that we expect.

When the perceptron is enough trained, we can provide it some inputs and check if the answer is right. In such case
its "weights" have been properly adjusted and the perceptron has learned :-)
regards, saludos

Antonio Linares
www.fivetechsoft.com
User avatar
Antonio Linares
Site Admin
 
Posts: 41205
Joined: Thu Oct 06, 2005 5:47 pm
Location: Spain


Re: Artificial intelligence - Class TPerceptron

Postby Antonio Linares » Tue May 16, 2017 4:47 am

Image

Pedro Domingos: The Master Algorithm
https://youtu.be/95XgpJAX6YE

Image

Image
regards, saludos

Antonio Linares
www.fivetechsoft.com
User avatar
Antonio Linares
Site Admin
 
Posts: 41205
Joined: Thu Oct 06, 2005 5:47 pm
Location: Spain

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 2:59 pm

Antonio,

I just tried compiling your TPerceptron example code and I am getting 66 errors. The first one is:

Perceptron.prg(40) Error E0016 Syntax error: ' '

At this line:

local oPerceptron := TPerceptron():New( 2 )

There are a lot of these same errors (on different lines).

I can't see anything wrong with the line. I note that I have started seeing this error message in my own code recently too. Maybe I am missing an include or something. Any ideas?

Regards,
James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 3:12 pm

Antonio,

There are only two other types of errors, "ambiguous reference..." and those like this:

Perceptron.prg(101) Error E0019 #error: 'Method "Learn( aInputs, nExpectedOutput )" not declared in class: TPerceptron'

None of the error messages are valid so I can't imagine what is triggering them.

James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby ukoenig » Thu May 18, 2017 4:12 pm

James,

That was a nice test of the practical use of my sample-collector
I included the sample from Antonio 1 : 1 to a free section.
I didn't change anything. NO errors ( just 5 minutes time to test it ) .

The xBrowse-results :

Image

Sample included in GROUP 3 SECTION 3 ( Dialog-section )

Code: Select all  Expand view

#include 'fivewin.ch'
#include 'xbrowse.ch'

// --------------------------------------------------------------
// -------------- GROUP 3 with DIALOG ----------------
// --------------------------------------------------------------
// --------------------- SECTION 1 -------------------------

FUNCTION GRPS3_SEC1()
LOCAL oBar

DEFINE BUTTONBAR oBar OF oDlg SIZE 56,64 2013 HEIGHT 96

oBar:oGrpFont  := oMedium

DEFINE BUTTON OF oBar PROMPT "Back"    RESOURCE 0x100DF GROUP LABEL "Navigate" COLORS CLR_WHITE,CLR_HBLUE
DEFINE BUTTON OF oBar PROMPT "Top"     RESOURCE 0x100E9
DEFINE BUTTON OF oBar PROMPT "Bottom"  RESOURCE 0x100EA
DEFINE BUTTON OF oBar PROMPT "Add"     RESOURCE 0x2002F GROUP LABEL "Edit" ;
      COLORS CLR_WHITE, { { 1, CLR_BLACK, CLR_WHITE }, .F. }
DEFINE BUTTON OF oBar PROMPT "Edit"    RESOURCE 0x20022
DEFINE BUTTON OF oBar PROMPT "Delete"  RESOURCE 0x20033
DEFINE BUTTON OF oBar PROMPT "Save"    RESOURCE 0x1003C GROUP LABEL "Export" COLORS CLR_WHITE,CLR_GREEN
DEFINE BUTTON OF oBar PROMPT "Refresh" RESOURCE 0x30050
DEFINE BUTTON OF oBar PROMPT "Setup"   RESOURCE 0x1005D BTNRIGHT

AEval( oBar:aControls, { |oBtn| oBtn:bColorMap := { |o| If( o:lMOver, { CLR_BLACK, CLR_HRED }, nil ) } } )

RETURN NIL

// --------------------

FUNCTION GRPS3_SEC2()
LOCAL oBar

DEFINE BUTTONBAR oBar OF oDlg 2013 SIZE 56,56 HEIGHT 80

oBar:oGrpFont  := oMedium

DEFINE BUTTON OF oBar FILE c_Path1 + "top.bmp" PROMPT "Top"  GROUP LABEL "NAVIGATE"
DEFINE BUTTON OF oBar FILE c_Path1 + "goto.bmp" PROMPT "GoTo"
DEFINE BUTTON OF oBar FILE c_Path1 + "bottom.bmp" PROMPT "Bottom"
DEFINE BUTTON OF oBar FILE c_Path1 + "new2.bmp" PROMPT "Add"  GROUP LABEL "EDIT"
DEFINE BUTTON OF oBar FILE c_Path1 + "edit.bmp" PROMPT "Edit"
DEFINE BUTTON OF oBar FILE c_Path1 + "delete0.bmp" PROMPT "Delete"
DEFINE BUTTON OF oBar FILE c_Path1 + "excel.bmp" PROMPT "Excel"  GROUP LABEL "EXPORT"
DEFINE BUTTON OF oBar FILE c_Path1 + "pdf.bmp" PROMPT "PDF"
DEFINE BUTTON OF oBar FILE c_Path1 + "printquick16.bmp" PROMPT "Print"
DEFINE BUTTON OF oBar FILE c_Path1 + "exit2.bmp" PROMPT "Exit" GROUP BTNRIGHT

RETURN NIL

// --------------------- SECTION 2 -------------------------

FUNCTION GRPS3_SEC3( oFld3, nSavePage )
local oPerceptron := TPerceptron():New( 2 )
local n, nX, nY
   
// the perceptron learns nAnd()
for n = 1 to 1000
      oPerceptron:Learn( { nX := nRandom( 1 ), nY := nRandom( 1 ) }, nAnd( nX, nY ) )
next
   
MsgInfo( oPerceptron:CalculateOutput( { 1, 1 } ), "1" )  
MsgInfo( oPerceptron:CalculateOutput( { 1, 0 } ), "0" )
MsgInfo( oPerceptron:CalculateOutput( { 0, 1 } ), "0" )
MsgInfo( oPerceptron:CalculateOutput( { 0, 0 } ), "0" )

// We create a new perceptron to learn nOr()
oPerceptron := TPerceptron():New( 2 )

for n = 1 to 1000
      oPerceptron:Learn( { nX := nRandom( 1 ), nY := nRandom( 1 ) }, nOr( nX, nY ) )
next
   
MsgInfo( oPerceptron:CalculateOutput( { 1, 1 } ), "1" )
MsgInfo( oPerceptron:CalculateOutput( { 1, 0 } ), "1" )
MsgInfo( oPerceptron:CalculateOutput( { 0, 1 } ), "1" )
MsgInfo( oPerceptron:CalculateOutput( { 0, 0 } ), "0" )
   
XBrowser( oPerceptron )

RETURN NIL

// ----------------

CLASS TPerceptron

DATA aPreviousWeights
DATA nPreviousThreshold
DATA aWeights          
DATA nThreshold        
DATA nLearningRate    

METHOD New( nInputs )

METHOD Learn( aInputs, nExpectedOutput )

METHOD CalculateOutput( aInputs )

ENDCLASS

// ------

METHOD New( nInputs ) CLASS TPerceptron
local n

::nPreviousThreshold = 0
::aWeights = Array( nInputs )
::aPreviousWeights = Array( nInputs )
::nThreshold = 0
::nLearningRate = 0.1

for n = 1 to nInputs
   ::aPreviousWeights[ n ] = 0
   ::aWeights[ n ] = 0
next  

return Self

// ----------

METHOD Learn( aInputs, nExpectedOutput ) CLASS TPerceptron
local nError := nExpectedOutput - ::CalculateOutput( aInputs )
local n
   
for n = 1 to Len( ::aWeights )
      ::aWeights[ n ] = ::aPreviousWeights[ n ] + ::nLearningRate * nError * aInputs[ n ]
next
   
::nThreshold = ::nPreviousThreshold + ::nLearningRate * nError

 ::aPreviousWeights = ::aWeights
 ::nPreviousThreshold = ::nThreshold

return nil

// ----------

METHOD CalculateOutput( aInputs ) CLASS TPerceptron
local nSum := 0
local n
   
for n = 1 to Len( ::aWeights )
      nSum += aInputs[ n ] * ::aWeights[ n ]
next  
       
nSum += ::nThreshold

return If( nSum > 0, 1, 0 )

// --------------------- SECTION 4 -------------------------

FUNCTION GRPS3_SEC4( oFld3, nSavePage )

RETURN NIL
 


regards
Uwe :roll:
Last edited by ukoenig on Thu May 18, 2017 4:49 pm, edited 1 time in total.
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
ukoenig
 
Posts: 4043
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 4:39 pm

OK, I found the problem!

It only occurs when I copy code from the forum. Sometimes there are special characters (invisible) instead of spaces indenting the code. I this case it was this line:

Code: Select all  Expand view
CLASS TPerceptron
   METHOD New
[special characters]METHOD Learn
ENDCLASS

 


When I changed the invisible characters to spaces there are no more errors.

I haven't figured out which characters they are as it is difficult to do with hex view in my editor. At first I thought they were tabs, but I inserted tabs and they work fine.

James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 5:15 pm

Ok, normally you use 3 spaces to indent. These are hex 20,20,20.

Some of the lines in the Perceptron code indented with spaces but others are indented with A0,20,A0. This causes a syntax error by the compiler.

I am at a loss as to how code that was working, can then be uploaded to the forum and end up with the A0,20,A0 syntax. I have never seen this until recently. Perhaps the forum software is doing this and it wasn't before?

Regards,
James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby ukoenig » Thu May 18, 2017 8:08 pm

James,

I copied from the posted code and did a past with my editor.
Maybe Your used EDITOR is the problem ?

regards
Uwe :?:
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
ukoenig
 
Posts: 4043
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany

Re: Artificial intelligence - Class TPerceptron

Postby Enrico Maria Giordano » Thu May 18, 2017 8:48 pm

The problem is a bug in MS Edge. Try with IE and it will work fine.

EMG
User avatar
Enrico Maria Giordano
 
Posts: 8243
Joined: Thu Oct 06, 2005 8:17 pm
Location: Roma - Italia

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 9:35 pm

Enrico,

The problem is a bug in MS Edge. Try with IE and it will work fine.


Thanks so much for that tip. I am using Edge. I'll try to remember the next time I copy text to use IE.

I have been using Edge for a couple of years now and I never had that problem before. I suppose the bug must have been introduced recently.

Regards,
James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby James Bott » Thu May 18, 2017 11:15 pm

Antonio,

Pardon me for accidentally hijacking your thread.

I have found a good article about preceptrons:

Perceptrons - the most basic form of a neural network

James
FWH 18.05/xHarbour 1.2.3/BCC7/Windows 10
User avatar
James Bott
 
Posts: 4840
Joined: Fri Nov 18, 2005 4:52 pm
Location: San Diego, California, USA

Re: Artificial intelligence - Class TPerceptron

Postby rhlawek » Fri May 19, 2017 3:03 am

I've been looking for some old source code to prove it to myself but this looks very similar to what I was taught as Predictor/Corrector methods back in the mid-80s when I was working on my mechanical engineering degree. I had forgotten about it until reading this code. If I remember correctly, for learning purposes, we used it for contour mapping, as in topographical maps. I remember it being fun to work with, but for the life of me I can't recall how it actually worked. I would need to find the original source to be sure. I may only have it on a floppy disk, probably written in Fortran 77.

I did do some derivative work based on that method, written for an aerospace company, which means nothing I owned or can share as I wouldn't have a copy. I suppose the original code I wrote it in college I own. If I can find it I'll see about porting it to harbour and will share it. Don't wait for it, it may be on a 5 1/4" floppy disk, and I don't have one of those drives anywhere any longer.

Robb
User avatar
rhlawek
 
Posts: 193
Joined: Sun Jul 22, 2012 7:01 pm

Re: Artificial intelligence - Class TPerceptron

Postby Enrico Maria Giordano » Fri May 19, 2017 8:03 am

James Bott wrote:Enrico,

The problem is a bug in MS Edge. Try with IE and it will work fine.


Thanks so much for that tip. I am using Edge. I'll try to remember the next time I copy text to use IE.

I have been using Edge for a couple of years now and I never had that problem before. I suppose the bug must have been introduced recently.

Regards,
James


Yes, I saw it the first time with W10 Anniversary Update, if I remember correctly.

EMG
User avatar
Enrico Maria Giordano
 
Posts: 8243
Joined: Thu Oct 06, 2005 8:17 pm
Location: Roma - Italia

Next

Return to AI Introduction (Harbour code and samples)

Who is online

Users browsing this forum: No registered users and 7 guests