Captain's Log
Hi,
I have exciting (over anticipated) news -
the first run of the embodied evolution experiment took place
overnight with a population of 4 (starting from all weights=0).
the robots have converged on a sub-optimal solution - but a solution
none-the-less.
the experiment was run for 13 hours - but I dont know how long it took
to get to this solution.
the evolved behaviour is a "spinaround-approach" - nothing like the
"swagger-approach" we designed by hand - interestingly, this is one of
the common behaviours that Miguel observed in simulation runs (I had
thought it unlikely that that particular method would be exhibited in
the real robots - but there it is).
There do not appear to be any futher reproduction events happenning
since they all get to the light in reasonable time - nobody is weak
enough to accept any (mutated) weight values from anyone
else. Although I'll let it run a while longer to see if anything
changes.
Cant get video Im affraid because there's something wrong with the
camera on sphene. And I've not yet debugged the routines that collect
weights or data during the run - so the results are purely subjective
:)
I should also declare for, Jordans sake, (though I expect he's stopped
reading by now) the true nature of the task - the robots use a 4
weight [-7,8] FF NN with _one_bit_ of input (1-sensora brighter than
sensorb,0-otherwise) and one bias node. Thats it - they are basically
given the input 'left'/'right' and they have to wire that to the
motors the right way around. Despite this there are many and varied
interesting behvaiours that arise during the course of the
experiment. (I found the "two steps forward one step back" behaviour
most amusing).
thats the end of the news (except that tupper one seems to be purely
on battery:( ),
before I start fucking about with changing the reproduction
probabilities (to try and get it out of local optima like in the result
of run 1) I wanted to check that Im doing the same as you in the
simulation. It may just be because the population is too small, of
course.
do you have a 255 ceiling on the energy/fitness levels or can they increase
indefinately in your simulation?
Im using the following alg.
when you get to the light - if fitness > 127 then fitness = 255 else
fitness = fitness + 128.
weight values are sent whenever random <= fitness and when this happens
fitness=fitness -1 (fitness is not decreased at any other
time). minumum value of fitness is 10. (which is what it starts with BTW)
if a weight value is recieved it will NOT be accepted if fitness >
(random / 2)
i.e. resistance is greater than horny-ness - right?
is that what you did?
I'll try another run and we'll see what happens.
1) there is a problem with the backup circuit: it appears that when running on battery alone (but through the backup circuit) the robot will cut out (and therefore stop running). This appears to be because the chip on the circuit cannot deliver enough current to handle the motors changing direction (except perhaps when the battery is brand new?). Anyway whatever the reason, the robot cut out about 5 times in one hour (it wasnt even connected to the floor). This appears to be an artifact of the backup circuit because if we just wire up the pic direct to the battery (and still ignore the floor) then using the same battery it has not stopped once in an hour.
2) the backup circuit is completely unnecessary anyway - we can use the floor and battery wired in paralell (see *) and it when the floor is in contact it will drain a negligible amount of power from the battery (see **) Giovanni says that he only built the circuit that Paaras designed because we asked him to :)
3) This news became apparent because Giovanni mentioned that the chip for the rechargable battery does not control whether the battery is connected to the load (like our current circuit does) rather the battery and the power supply are permanently connected in paralell to the load - it merely determines how to charge the battery. So when we switch to rechargable batteries we will not suffer from the cutting-out problem.
4) We are currently running both a battery_only test and a battery_and_floor_without_backup_circuit test to see how long they last. (Though I may interrupt these in favour of doing another overnight evolution trial because there's no point in doing a continuous battery test when Im not here to see it stop :)
* with a single diode to protect the battery from being 'charged' by the floor and busting its seams and leaking gooby stuff all over the place. and you still need the retifying diodes, of course.
** approx. 1 mA in normal usage, and still less than 20 mA when the motors are restricted; as opposed to 60 mA in normal usage and 240 mA when the motors are restricted which is what you use from the battery when the floor is not in contact
So, the current code says that
if ((random / 2) < fitness) then do NOT accept received weights
this means that it cannot accept a weight if its fitness is over 127,
which is bad.
I'll change this to
if (random < fitness) or (random < fitness) do NOT accept weights.
that should reduce the probability of acceptance yet still make it
possible to accept when fitness is high.
I suppose I could do
if (random < (fitness + (255 - fitness)/2)) do NOT accept.
run 4 (13hrs) has found a non-spinning solution thats much more direct :)
Im starting to characterise the behaviours according to their two components ie. the trajectory when light is on left and the trajectory when light is on right. We can call the one that gives the most forward translation the primary arc and the other the secondary arc.
so, for example, in run 1, the primary arc had a diameter of about 3 inches whereas the secondary was almost on-the-spot, both arcs had the same angular sign/ both rotated the same direction.
now I can tell you that in run 4 the primary arc is about 18 inches diameter and the secondary arc is on-the-spot but in the opposite direction (like a lopsided version of the designed 'swagger' solution.
Though I note that most of the time they dont get as far as invoking the secondary arc before a turnabout kicks in. I adjusted the turnabout sensitivity - probably a mistake. But I think this may also be affected by the fact that the room lights are off - that means that unless the light enters or leaves the field of view then whatever else you are looking at is consistently black.
the behaviours are not consistent and their is still reproduction going on. its not true to say that they are all clearly doing the right thing. though they do seem more consistent now I';ve turned the lights on !
observation: when the batteries are weak, but still strong enough to move the robot, and the robot gets stuck against the wall it doesnt invoke the turnabout behaviour like it should. My geuss is that with the motors straining and the battery struggling to deliver enough voltage the sensor readings fluctuate (even though there is a very consistent amount of light hitting them). In these cases the robot has to wait until another robot nudges it. But in the meantime the battery gets even weaker...
BTW. the one without the backup circuit seems fine so far...
There are (at least) 5 easily identifiable ways in which the pics/out current setup are/is limited. And for many of them there are several easy things we could do to make it more interesting/less limited. the emphasis of these suggestions/thoughts is supposed to be for use with the pic with minimal alterations - stuff we could do without a new platform to make the experiment more interesting.
Input deprivation
-use raw inputs
-use 'moving window' of past sensor inputs
-use ir as a 'proximity test' ie. other robot is nearby
-multiplex sensors (there are two 'select' pins, one for each sensor, that could be used to define 4 sensor 'selects' with the addition of a couple of logic gates. but this would require modification of the pic assembler. - 'multiplex' binary sensors (eg bump) using series resitors
output lmitations
-transform one output into 'turn amount' and assume a constant (combined) speed.
-use the beeper output to drive something else
-use one of the sensor select lines as an output (loose a sensor :)
reproduction bandwidth
-we could squeeze a few more weights through with the current protocol (ie wieght-index and weight value in same byte) if we tried - there's a large portion of the byte range reserved for communication of weights to mother but actually they are received serially in order so it only needs a range of 16 (not 64) values for this task. that might get us maybe 8 weights instead of 4. more than that and we'll have to use something different.
- the next most simple thing is to use a two-byte protocol (per weight) one byte for the weight index and the second for the weight value (perhaps a third as some kind of check sum to make sure that the index and the value that you got were a pair).
- use an encoding scheme for the weights. maybe we could use an IFS as the encoding - a 1D IFS requires 4 weights, then we could use the attractor to define the weight values... somehow.. lets see, we could produce the attractor by non-deterministic itteration from a random start position, store all the visited points (after the first 20 to allow settling in to the attractor). Then we need some way to map the attractor onto a set of weights. see separate text on using the IFS as Jordan intended (which I havnt written yet)
slow computation
in general it is necessary to use a small number of significant computations rather than a large number of small adjustments - eg. continuous time RNNs are out. Unless, see next point
- re-code the pics with a dedicated network simulation main loop. this would definately run fine you can get the weight values into ram variables instead of eeprom variables. but even if the itteration variables and the accumulating sum are ram variables and the weights are still out in the eeprom it may be fine - certainly at least an order of magnitude faster than it is now. At the very least you can provide some user variable that are in ram so that its not the case that every single assignment to a variable takes at least 16 clock cycles!
- use external hard-wired dynamical system ie. digital pots. they come with at most 4 on a chip so its not pratical to use too many weights (eg 8 node fully connected network=64 weights=16chips :( ) but maybe we could have just twelve (thats only 4 chips) we could do the chip select with only two bits :) Then all we have to do is find a way to use the resultant IFS to control the robot... see separate text that I still havnt written
task/ environment
so, supposing we can wriggle out an extra sensor and maybe an extra output and maybe a better/faster control architecture that we can still communicate... what would we do with it?
- put lights (or ir senders) on the robots and do some kind of P+E
- allow these lights to be controlled and do some kind of behavioural communication
- soccer - put a light on a puc
- use a directional ir so you can tell the direction of other robots (and other beacons and stuff) then maybe do approach, alignment, avoidance... something
- other cool stuff
so, when its running on battery only and the motors reverse from full power thisway to full power thatway it cuts out. (even if the battery is new) This only happens if you're using the recharge circuit. And it doesnt appear to happen if the motors are stopped forcibly before reversing eg. hit a wall then reverse. Its that back emf isnt it. It also doesnt appear to happen if you've been in contact with the floor recently (within a half second?) but Im not atall sure about this and its difficult to arrange a test. IF this is true it could be because the cap is still driving and it can deliver the necessary current... this could be bullshit. Anyway, this doesnt happen often in practise UNLESS you're control architecture produces numerous reverse commands (which my recurrent version of the network does :( ). It also never happens when powered directly by the battery rather than through the backup circuit. So, this shouldnt be a problem for the recharge circuit.
My plan is to use a cap on both sides of the backup circuit until we get the rechargeable circuits. My hope is that the cap will charge-up when there's power available (ie anytime when the motors arent reversing) and supply extra support when they do reverse. We'll see.
so, when its running on battery only and the motors reverse from full power thisway to full power thatway it cuts out. (even if the battery is new) This only happens if you're using the recharge circuit. And it doesnt appear to happen if the motors are stopped forcibly before reversing eg. hit a wall then reverse. Its that back emf isnt it. It also doesnt appear to happen if you've been in contact with the floor recently (within a half second?) but Im not atall sure about this and its difficult to arrange a test. IF this is true it could be because the cap is still driving and it can deliver the necessary current... this could be bullshit. Anyway, this doesnt happen often in practise UNLESS you're control architecture produces numerous reverse commands (which my recurrent version of the network does :( ). It also never happens when powered directly by the battery rather than through the backup circuit. So, this shouldnt be a problem for the recharge circuit.
My plan is to use a cap on both sides of the backup circuit until we get the rechargeable circuits. My hope is that the cap will charge-up when there's power available (ie anytime when the motors arent reversing) and supply extra support when they do reverse. We'll see.
it turns out that the problem of mother not being able to see the crickets was really a problem of the crickets not being able to see mother. The confusion arrose because we knew that the robots must have mother in view (else they wouldnt have stopped to call her) but they couldnt see her request for weight data because of the timing of the bytes that mother sent. The problem was that other robots could write-over the request byte with their own calls to mother. So, I changed the waittoseemyid proceedure so that it waits increasingly long intervals between calls to mother. This should mean that any two robots get out of phase so that one doesnt keep over-writing the others ir values.
It works significantly better now :)
after helping us complete their construction and our preliminary experiments they get to use the robots for interesting things.
the group at Reading has some interesting stuff
The group is currently committed to creating a networked robotics environment which can be accessed from anywhere on the Internet, providing interactive robotics experiments.
D.A.Keating is the guy that Jordan met recently who told him about the floor they built for the British Science Museam.
In one of my afternoon hacks, I made a Java version of the tupperbot simulation (first amazing fact: It works faster than on the onyx with 5 processors!). Well, things didn't work initially as expected (the tupperbots almost never converged to the appropriated weights, as it did in the first simulation (lets from now on call the first simulation S1, and the Java simulation S2)). The only thing that changed from S1 to S2 was the size of the arena, the size of the robots, and sensing functions. The rest was exactly the same: Learning algorithm, kinematics, etc.
After hours of stupidly watching these fucking bots, I have the theory that the learning algorithm likes to stay in local optimum (like the spinning behavior to get to the light), if there are regularities in the environment that can be exploited. For instance, the spinning behavior towards the light, made the robot go to the walls. Once the robot was close to a wall, it started a "wall following" behavior, that slowly directed the robot towards the light (in one of the spins it touched the light).
Another factor I think is important, is that in S2 there is no
random sensing like in S1 (in S1 I simulated some kind of range
sensing which had a lot of noise). Richard's algorithm makes the robots
turn around randomly when the light readings are exactly the same,
introducing some more (healthy) randomness into the process. in this
sense, "real world tests" and S1 had some randomness built-in in how
the agents perceived the world.
My conclusions are:
I have observed something a bit strange, but I'm sure it's actually very normal -- though the power supply is sending out 12.5 volts, the voltage that's actually getting to the recharge circuit is only about 10.8 volts at most. Much of the time it is a lot less because the capacitor is discharging a lot of the time. So, I don't quite understand why the voltage coming out of the rectifier is that low. Right now the battery is very low -- about 1.5 volts or so.
Since contact with the floor is lost so often (due to the oxidation) the capacitor is doing quite a lot of work in smoothing the voltage. Though the voltage coming out of the capacitor is almost always higher than needed to power the pic, it seems that it is never, or seldom, high enough to recharge the battery. Does this mean we need to raise the voltage to something much higher? 24 volts? If so, the capacitors are rated only for 16 volts. Also, if we raise the voltage, we will need to adjust the resistor values on the motors. Anyway, let's talk about this. We have group meeting today from 1:30 to about 3:30. Any time before or after is fine with me -- you're welcome to come to the meeting itself, if you would like to.
Prem started another test last night and the good news is that over 12 hours later the beacon is still running and collecting data. Only two of the robots were still functioning as of noon today -- the rechargeable bot seems to require a higher input voltage, and the battery of one of the others was dead. The other two are running fine.
We still have problems with the interrogation protocol, but I think we can improve it.
I started an experiment at 12:40a.m.
Earlier I ran a small trial for about 70 minutes. The data that I get has
incomplete or incorrect records about 20-30% of the time.
So I ran a test with Richard's logo code and found that it reports bad
data 40-45% of the time.
I guess we will need to modify the cricket code. It would also be better
to have the cricket report its fitness first, as it is more important.
I'm now maintaining 2 data files, one for visual inspection of data and
one for analysis. (The default names of these files are "check" and
"matfile").
I wanted to use Matlab on the other Mac, so I was writing the file
directly to the other hard disk ("Development" on the 8500), but I can't
read the file until it is closed. I can, however, view the file on the Mac
I'm running the code from. This might be due to the fact that the 8500 is
running an older version of MacOS than the 7600. Anyway I'll look into
that later.
I downloaded the code and tried again, but the motor didn't work. Giovanni took a look at it and he said that the chip controlling the motor isn't working and will need to be changed. He said if we have a broken PIC board he can just replace the chip, otherwise we should replace the whole board. I'm not going to be in the lab tomorrow, so I'll send Sevan a message and let him know. - Prem
Ok. My guess is the following: The chip that does't work is the H-Bridge. The reason for that is that it had to deliver to much current. As far as I know, and mostly because of the rechargable circuit, the current voltage is around 13.5 Volts. Is it possible that the PICs can't handle so much? I've had some bad experience with H-Bridges (I burned a couple of them already).
If I'm right, we should consider new alternatives for the new backup circuit, instead of increasing the voltage.
And I also might be wrong (everything is possible).
I've just ordered 30 light sensors of the type specified on the cricket's web site. Also I ordered 6 of a slightly different variety that has a larger resistance range (50k-200k, instead of 100k-200k). The 6 will be here soon, but the 30 are on back order until July or August, so we might have to make due with the less sensitive light sensors that we get from you-do-it. We might find, however, that the 6 sensors that will arrive next week will be suitable substitutes -- let's see what happens.
Richard, I know you mentioned that Fred's site specifies the IR components used, but I can't find it... Giovanni has some guesses as to what might work. In any case, we have enough pairs for our current needs.
I stopped the test you had going and started up a new one, just looking to see what happens with a single bot and what kind of communication failures we get.
The asymmetry between the IR setups on the cricket and mother probably have something to do with the problem, but I still don't know how the cricket can know to send its ID, and mother can know to send an acknowledgement, but the network specs don't make it over... could it be a timing issue? On the other hand, it could have more to do with the location of the cricket -- some places around the beacon have good "sight lines" and others don't. We can try adding a third pair of IRs onto the beacon. Maybe that will do the trick.
In a "coffee break", Giovanni suggested many interesting things (as usual). I think that we should think about them for the future. And future means when we are all set with the experiments, have written the tech report, and are able to ask for billions (lets say many thousands) of dollars.
1-) Replace the aluminum strips in the floor for steal. Giovanni says that we would not have the problems we are having right know (the floor is sadly and slowly falling apart), because steal much harder. I thought that steal would be much thicker but, once again, I was wrong. We can select the thickness that we want. I don't know (question for Sevan) if the company that supplied us with aluminum foil has also steal foil...
2-) Check the new Motorola chips and DSPs. Paaras is thinking to build (in the future) our next generation of robots, based on one of these micro-controllers. Actually, the board would almost the size of the pics, put 1000000000000 orders of magnitude more powerful.
I think that these are important things we have to slowly think about, and ONCE we are finished with these experiments we should quickly move forward.
Steel Tape: 1-1/2" x 20-ft. roll Each $7.49
It seems expensive....
Hi. The battery recharger is having a difficult time due to poor floor contact. The overall state of the floor is probably much degraded from when it was new --
the robot contacts are beginning to require frequent cleaning. Giovanni found a potential source for steel tape. I'll follow up on that. In any case, I suspect that the recharger will work much better on steel.
did I get everyone? thats quite a team we have :)
hello! - I'm back!
Im very pleased to see so much robot-team activity. Thanks everyone.
I've got some questions/comments from the past emails
(perhaps they're out of date already - you can tell me):
All:
SAB demo; I finally got something about this. havnt read it yet.
Paraas:
hows the beacon longevity these days? has it only run for 12 hours on
one occasion?
Giovanni:
Does the recharge circuit use the 'sense the voltage and switch on the
battery' circuit afterall or is the cell in parallel to the floor in an 'always
connected' kind of way?
Paraas:
did a cricket die :( Is it confirmed that the increased voltage
(13.5) is too high?
Miguel:
re - problem with back-up behaviour - spinning to the light and then
backing-up into it. I think that doing the
opposite motor output for one step may still result in finding the light
whilst facing the wrong way. If its necessary to fix this atall -
perhaps spinning to face in the brightest direction, then backing up
will work. Or perhaps there's a way to use a few random turns and a
few straight runs to make it very unlikely that your stuck against the
light when you restart. [I'll check out S2 in a minute :) - cool]
Sevan:
ir components from Freds pages -
http://fredm.www.media.mit.edu/people/fredm/projects/cricket/internal/mscricket2.0.html
and the suppliers -
http://fredm.www.media.mit.edu/people/fredm/projects/cricket/suppliers.txt
perhaps you couldnt find them because I havnt told you the password
for the internal and sponsor only pages. This is to be protected and
used wisely and all that - but I reckon its cool for everyone in the
robot group to know it - though I wont put it here in case the
captains log ever becomes public.
Sevan:
I agree that the mother communication problem may well be a timing
issue. although it could be because the problem might be that
they dont stop immediately and they are necessarily on the boundry of
where they can see mother (ie they see mother and then move just a
little bit and then they can see her).
we could put a built-in 'go to light' beh to get them closer to
mother after they get first contact ? this would also make the back-up
beh more reliable :)
anyway - we need to devise a way to properly test whether its a
timing issue or not.
all:
agreed that we should think about the future and move on as soon as the
initial experiments and tech reports are done (though that might not
happen instantly :) Glad to see that prem is
working on the next generation. "1000000000000 more orders of magnitude"!?
10^(10^12) ... cool.
sevan:
so this supplier really has adhesive stainless steel tape?
giovanni:
I look forward to hearing the dirt on the recharge circuit
difficulties. ('the dirt' english colloquialism meaning derogatory
gossip details, usg.'whats the dirt on that cute new undergrad?')
glad to be back in the thick of it :)