Hi there my non-existing readers. Today I will tell you a little about one of my projects that I working on in my free time. It's based on Erlang and it represents a simulation of a neural network. I will not bore you with programming details or specifics, but I will try to describe the whole idea, concept and what observations I have on this topic.
So what I am building?
I am building what is called a neural network. This represents a set of entities, that store one "memory" and are linked between them. The system has one entry point from which the memory chain starts. All the memories are connected between them based on the "last action" principle. So a "memory" will link to another memory that followed on the time line. All the memories are unique, so data will not be duplicated. To be sure this is happening, there is a process of searching a memory before it is created as a new one and linked to the chain. So this could generate self references or closed chains of memories.
How do I search, without infinite loops?
Initially I had this problem: when searching for a memory I was getting into a endless loop. "Why?" you would ask. So let me tell you how searching works. The entry point neuron is asked if he contains the searched memory. If not, he will ask the next linked neurons. These neurons will check their content and ask each individually their connections, until the neuron that contains the data is found, and send a message back to the "requester". All the other search chains dies if the end neurons of the chain has no connections. But, in some cases the search can go in circles for ever. So at first I added a TTL or Time To Live mechanism, which counted how many levels will be sent in the neuronal structure. When each neuron passed the information to the next neurons it decreased the TTL with one. So after a number of steps, the search will die, and the searcher after a defined timeout will consider that the search has failed and create a new memory.
After some tests, I realized that I could optimize this process. So I added to the message sent for the search, a list of neuron nodes, that this message had passed through. This way, if a neuron find itself in this list, it means that the search had already passed through him once, and it will close this search chain. In some cases some neurons could close several search chains, depending on how the data is stored and linked. Because a node can be referenced by multiple other nodes and can be part of several neuronal circles.
At this moment this is my state of the project. But what will be done next?
- Create a receiver. This will be an element, that will have a fixed type of input that will create memories, that will be passed to the neuronal network. Just to understand what this is, it might be a light detector that sends 1 or 0 if light is on or off, or a news parser that reads news or texts from a webpage and so on..
- Create a trunk. This will be an element that will condense memories from several receiver if needed. For example having multiple light detectors, this trunk will condense them in one list of values, or like a small image. This is very helpful when you might want to simulate an eye or the tactile sense of a finger.
- Create a processor. This will be an element, that will try to abstract memories, and will try to see if memories are alike, similar and set a grade for this comparison. This will offer the possibility to look for similar memories.
- Tandem memories. I will have to figure it out, how I can store multiple memories that happens in the same time, but from different receivers. This will be a very critical step, because most of the human behavior is developed around this.
Let's talk a little about Tandem Memories. I don't know if you ever try to analyze your own brain functions. If you, for example look at a tree and it starts moving from left to right, you brain will react by telling you that the wind is blowing and you might have a preemptive feeling that the wind will arrive any moment. This is done by associating the image / set of images with the wind feeling on your skin, event that happened since you were born. Same effect can be observed with music and events. I used to listen a specific band every time I was riding my bike. Now if listen to that band, I will remember parts of images that I've seen during my biking. This also can apply vice-versa, if I bike, in similar conditions that I did in the past, parts of songs or lyrics will come to my mind.
Stay tuned. To be continued ...