Re: A followup on Concurrency within Python

MichaelSparks — Thu, 20 Mar 2008 11:59:46 -0000

Hi there,

You may find Kamaelia interesting - I've been adding clean multicore support recently. You can find an overview of its design philosophy on our summer of code page here: http://kamaelia.sourceforge... (linking there because there's an embedded presentation & lots of links).

You can find discussion of how multicore support was added here:
http://yeoldeclue.com/cgi-b...
Rather than pyprocessing it uses pprocess since that seems sufficient. I'll take a look at pyprocessing though.

Single process pipeline:
Pipeline(
Textbox(position=(20, 340),
text_height=36,
screen_width=900,
screen_height=400,
background_color=(130,0,70),
text_color=(255,255,255)),
TextDisplayer(position=(20, 90),
text_height=36,
screen_width=400,
screen_height=540,
background_color=(130,0,70),
text_color=(255,255,255))
)

Multiprocess pipeline:
ProcessPipeline(
Textbox(position=(20, 340),
text_height=36,
screen_width=900,
screen_height=400,
background_color=(130,0,70),
text_color=(255,255,255)),
TextDisplayer(position=(20, 90),
text_height=36,
screen_width=400,
screen_height=540,
background_color=(130,0,70),
text_color=(255,255,255))
)

It just works. It seems to have similar inspiration as erlang, but I wasn't aware of erlangs execution model when I created Kamaelia. (It was more inspired by asynchronous hardware systems, unix pipelines, CSP and biological systems).

cf http://kamaelia.sourceforge...

I went into that (due to work modelling biological systems) with the assumption that whilst the primary communication would be akin to the nervous system (send and receive of pulses of information between processors) hence why the core of kamaelia is called Axon, that there would need to be an equivalent for a high latency, low use, but useful hormonal system. That boils down to a global key value store. (which will be migrating soon to an STM based store).

The interesting thing from my perspective is that this also mirrors a system from RAF Malvern from 30 years ago, called MASCOT which I heard about at christmas. That uses channels, instead of named inboxes/outboxes, and pools which are equivalent to our CAT ("hormonal" systems). MASCOT is utterly fascinating because they started from the same premise. I only recently heard about MASCOT (just before christmas), and the only reference I can find online is this, but if you scroll down, you'll find "The Official Handbook of Mascot : Version 3.1 : June 1987". Perhaps scarily, their first version appears to have been in 1975....)

http://async.org.uk/Hugo.Si...

OK, I'll get back to suggesting ideas for students for GSOC which are naturally highly concurrent, fun, in python beginner friendly and naturally multicore.

Incidentally, there is also another difference between python & concurrency in a functional language - the fact that objects in python are mutable. I blogged about some implications of this here:
http://yeoldeclue.com/cgi-b...

Kamaelia has proof of concept implementations in Java, C++ and ruby as well.

Have fun :-)

Michael.

Disqus - Latest Comments for MichaelSparks

Re: A followup on Concurrency within Python