Do they belong to you? Claim these comments.
Unregistered
aliases
- Michael Sparks
- Michael Sparks
- Michael Sparks
Michael Sparks
Is this you? Claim Profile »
4 months ago
in PyCon 2009: In ur brain, giving you the pythons on jessenoller.com comments
Sad*, but related fact: Kamaelia is a deliberate misspelling of "Camellia" (as in camellia sinensis - Tea), which was chosen because it's a shrubbery :)
... I'll get me coat.
... I'll get me coat.
1 reply
jnoller
Oh man, that's wrong
6 months ago
in jessenoller.com » Blog Archive » PyCon 2009 Talks accepted. on jessenoller.com comments
Congratulations :-) If you need anything from me, just shout. If you don't I'll look forward to reading the slides :-)
8 months ago
in jessenoller.com » Blog Archive » Hooray, Actors! on jessenoller.com comments
Downloadable (legally) from here:
http://dspace.mit.edu/handle/1721.1/6952
Your actual paper copy wins hands down though! :-)
I've been tidying up a download of Hewitt's 1973 paper (which appears to be OCRd and a bit dodgy in places) - very interesting reading. Turns out the Actor model comes from artificial intelligence and was actually an attempt to unify lots of different models of computation.
Oh, and lisp based. Should I be surprised ? :-)
http://dspace.mit.edu/handle/1721.1/6952
Your actual paper copy wins hands down though! :-)
I've been tidying up a download of Hewitt's 1973 paper (which appears to be OCRd and a bit dodgy in places) - very interesting reading. Turns out the Actor model comes from artificial intelligence and was actually an attempt to unify lots of different models of computation.
Oh, and lisp based. Should I be surprised ? :-)
1 reply
jnoller
I think everything is Lisp based. Everything >:|
8 months ago
in jessenoller.com » Blog Archive » Actors, concurrency and Kamaelia on jessenoller.com comments
Glad to hear it was useful :) Fully understand the time pressures - especially parental ones (have 2 small ones myself) :-) Also very much understand the motivation behind "I want to build one myself, to understand". Back at uni our lecturers very much liked quoting confucious at us "I hear and I forget. I see and I remember. I do and I understand". (this is very much the reason also behind the mini-axon tutorial :-)
For what it's worth, if you want any help/guidance through our code, just let me know. More than happy to help.
If you get a chance and would be willing, I'd be very curious to hear what you would view as a library approach rather than framework. I ask because, as noted, I think of Axon as a library & Kamaelia as a bucket of toys built using it, rather than as a framework. Also when Kamaelia hits 1.0.X I want to also move Axon to version 2.0.X. The core ideas that we want in Axon 2.0.X is the ability for people to cherry pick our stuff properly (Axon.Handle & Axon.STM are the beginnings of that) and so on. So any feedback (from yourself or anyone reading this) about how to make our system more usable in a pure a library fashion would be very useful.
But, if you don't have time/etc - no problem I've got similar pressures :) Also, I've been following up on the links you've been pointing people at, and will follow the links you've posted - these are much appreciated. (they're leading me down interesting avenues :) Also, I don't believe in one language, framework, or library to rule them all so any suggestions are welcome :)
Also, randomly, I agree about the need for an efficient interprocess messaging passing facility... It would be particularly useful for everyone...
For what it's worth, if you want any help/guidance through our code, just let me know. More than happy to help.
If you get a chance and would be willing, I'd be very curious to hear what you would view as a library approach rather than framework. I ask because, as noted, I think of Axon as a library & Kamaelia as a bucket of toys built using it, rather than as a framework. Also when Kamaelia hits 1.0.X I want to also move Axon to version 2.0.X. The core ideas that we want in Axon 2.0.X is the ability for people to cherry pick our stuff properly (Axon.Handle & Axon.STM are the beginnings of that) and so on. So any feedback (from yourself or anyone reading this) about how to make our system more usable in a pure a library fashion would be very useful.
But, if you don't have time/etc - no problem I've got similar pressures :) Also, I've been following up on the links you've been pointing people at, and will follow the links you've posted - these are much appreciated. (they're leading me down interesting avenues :) Also, I don't believe in one language, framework, or library to rule them all so any suggestions are welcome :)
Also, randomly, I agree about the need for an efficient interprocess messaging passing facility... It would be particularly useful for everyone...
1 reply
jnoller
Axon may very well be a library - conceptually, I am beginning to see Kamaelia as you do - a "Kamaelia as a bucket of toys built using it", it being Axon.
I think what might help (conceptually) is a further decoupling of Axon from the bucket of toys - I think the roadmap for 2.0 you're talking about is great, it's the difference between Zope, and the various zc.* packages they have moved to.
I will make time though - my primary point was to almost say "getting to it busy omg".
As for the MPI - I started exploring various implementations of those as well, I should post my notes on those. The nice thing is is that with multiprocessing I can swap out the serialization mechanism(s) and use MPI freely under the hood.
I think what might help (conceptually) is a further decoupling of Axon from the bucket of toys - I think the roadmap for 2.0 you're talking about is great, it's the difference between Zope, and the various zc.* packages they have moved to.
I will make time though - my primary point was to almost say "getting to it busy omg".
As for the MPI - I started exploring various implementations of those as well, I should post my notes on those. The nice thing is is that with multiprocessing I can swap out the serialization mechanism(s) and use MPI freely under the hood.
8 months ago
in jessenoller.com » Blog Archive » Guido answers your questions… on jessenoller.com comments
Are you aware that a complete mini-axon using generators is tiny and the rest is optimisations and extra stuff that you find useful in real world systems? By tiny, I mean this small:
* http://www.kamaelia.org/MiniAxonFull
A mini-axon using processes would be equally lightweight (shorter probably) and pretty awesome.
Also, it's easy to confuse the two halves of Kamaelia. If you think of Kamaelia as just an actor-type implementation, then it's actually more an actor-like implementation, with STM & an internal SOA system of just over 2000 lines (which is how big Axon actually is, excluding comments & docs), with 80,000 lines of examples...
However, personally I view it as a mechanism for building components which happen to be best used in a concurrent fashion. ie rather than viewing it as "a mechanism for using concurrency", I view it as "OK, assume we have concurrency, how can we use this to assist in building and maintaining systems". Axon also gives you the tools for taking these concurrent systems, and interfacing between concurrent systems and standard code. (http://www.kamaelia.org/AxonHandle)
As a result, I view Axon as a library which provides you with the tools wrapping up idioms useful for building collections of components which be a framework.
Anyway, potato/potato, tomato/tomato - if you like, you like, if you don't, you don't.
I'd love to replace our existing process based stuff btw with a multiprocessing based version though. If I was going to go down this route, I'd follow our mini axon tutorial to do so, largely becauseit's essentially the starting point I took with the multiprocess stuff recently and it worked out pretty well.
Beyond this basic stuff though, I've noted that people generally start talking about co-ordination languages and building up pattern repositories. The interesting intersection between these two which you get if you call things components rather than actors is it becomes natural to create components called a chassis. These chassis often instantiate directly in concrete usable form concepts that you'd normally refer to as a pattern - Pipeline, Graphline, Carousel, Backplane, Seq, TPipe, etc.
On a random note, you may want to check out MASCOT "Modular Approach to Software Construction, Operation and Test". I heard about it late last year, and it appears to have the same sort of architecture as Kamaelia. Interestingly (to me) it makes the same key decision - when you send a message outside your component, you don't know who is going to receive it. This then enables (and requires) a higher level system for connecting components together. The upshot is highly reusable components. This doesn't entirely surprise me - my ethos came from recognising that asynchronous hardware systems & network systems look strikingly similar... (cf http://www.slideshare.net/kamaelian/sociable-so...)
Anyway, reference for MASCOT: http://async.org.uk/Hugo.Simpson/ - skip down to the end of the page for this PDF: http://async.org.uk/Hugo.Simpson/MASCOT-3.1-Man... I was really pleased to be pointed at MASCOT, largely because it showed a large number of other domains where the same basic model has been used for well over 30 years... Just with non-existent exposure, and slightly different metaphors. Though we, like it, also have mechanisms for automatically visualising systems, with a 1:1 correspondence. Beyond that this also gives us a model that matches Edward Lee's "The Problem with threads" - we'd released running code long before that paper was published :-)
Anyway, I'm glad that you're looking at what we've done. If you use it, that'd be great, and I'll happily merge anything you'd like to have a life. (the only comment I'd make there is metaphors and accessibility count - this is surely the point of python? :-) If you don't take what we use etc but it helps you solidify your thoughts to "No, I don't want that, I want this", then likewise, I'm equally glad. If you do that, I'd love to know what you do try, since I like to merge best practice concurrency ideas into Axon :-)
I'd *REALLY* suggest looking at MASCOT though. Really made my Christmas last year when I was pointed at it.
We're currently having lots of fun using concurrency, primarily by allowing it to make our lives easier, and forgettable about :) It'd be nice to see something similar on top of multiprocessing (which we'll do if you don't, but it'd be great if you did - but I'd understand if your view was that you prefer a pure actor (sender knows receiver) model.
* http://www.kamaelia.org/MiniAxonFull
A mini-axon using processes would be equally lightweight (shorter probably) and pretty awesome.
Also, it's easy to confuse the two halves of Kamaelia. If you think of Kamaelia as just an actor-type implementation, then it's actually more an actor-like implementation, with STM & an internal SOA system of just over 2000 lines (which is how big Axon actually is, excluding comments & docs), with 80,000 lines of examples...
However, personally I view it as a mechanism for building components which happen to be best used in a concurrent fashion. ie rather than viewing it as "a mechanism for using concurrency", I view it as "OK, assume we have concurrency, how can we use this to assist in building and maintaining systems". Axon also gives you the tools for taking these concurrent systems, and interfacing between concurrent systems and standard code. (http://www.kamaelia.org/AxonHandle)
As a result, I view Axon as a library which provides you with the tools wrapping up idioms useful for building collections of components which be a framework.
Anyway, potato/potato, tomato/tomato - if you like, you like, if you don't, you don't.
I'd love to replace our existing process based stuff btw with a multiprocessing based version though. If I was going to go down this route, I'd follow our mini axon tutorial to do so, largely becauseit's essentially the starting point I took with the multiprocess stuff recently and it worked out pretty well.
Beyond this basic stuff though, I've noted that people generally start talking about co-ordination languages and building up pattern repositories. The interesting intersection between these two which you get if you call things components rather than actors is it becomes natural to create components called a chassis. These chassis often instantiate directly in concrete usable form concepts that you'd normally refer to as a pattern - Pipeline, Graphline, Carousel, Backplane, Seq, TPipe, etc.
On a random note, you may want to check out MASCOT "Modular Approach to Software Construction, Operation and Test". I heard about it late last year, and it appears to have the same sort of architecture as Kamaelia. Interestingly (to me) it makes the same key decision - when you send a message outside your component, you don't know who is going to receive it. This then enables (and requires) a higher level system for connecting components together. The upshot is highly reusable components. This doesn't entirely surprise me - my ethos came from recognising that asynchronous hardware systems & network systems look strikingly similar... (cf http://www.slideshare.net/kamaelian/sociable-so...)
Anyway, reference for MASCOT: http://async.org.uk/Hugo.Simpson/ - skip down to the end of the page for this PDF: http://async.org.uk/Hugo.Simpson/MASCOT-3.1-Man... I was really pleased to be pointed at MASCOT, largely because it showed a large number of other domains where the same basic model has been used for well over 30 years... Just with non-existent exposure, and slightly different metaphors. Though we, like it, also have mechanisms for automatically visualising systems, with a 1:1 correspondence. Beyond that this also gives us a model that matches Edward Lee's "The Problem with threads" - we'd released running code long before that paper was published :-)
Anyway, I'm glad that you're looking at what we've done. If you use it, that'd be great, and I'll happily merge anything you'd like to have a life. (the only comment I'd make there is metaphors and accessibility count - this is surely the point of python? :-) If you don't take what we use etc but it helps you solidify your thoughts to "No, I don't want that, I want this", then likewise, I'm equally glad. If you do that, I'd love to know what you do try, since I like to merge best practice concurrency ideas into Axon :-)
I'd *REALLY* suggest looking at MASCOT though. Really made my Christmas last year when I was pointed at it.
We're currently having lots of fun using concurrency, primarily by allowing it to make our lives easier, and forgettable about :) It'd be nice to see something similar on top of multiprocessing (which we'll do if you don't, but it'd be great if you did - but I'd understand if your view was that you prefer a pure actor (sender knows receiver) model.
2 replies
jnoller
Michael - wow thanks for this - when I get a chance I want to reply
with the reply a comment this detailed deserves
with the reply a comment this detailed deserves
8 months ago
in jessenoller.com » Blog Archive » Guido answers your questions… on jessenoller.com comments
Indeed, but we'd LOVE to have a version based on multiprocessing ...
9 months ago
in jessenoller.com » Blog Archive » What do you want to see in a concurrency talk… on jessenoller.com comments
OK, cool. Let's see.
Tell you what - how about I just package up the network components (no protocols, no pygame, no open gl, etc), along with a minimal set of other useful components as well by itself for you? ie the absolute minimum useful (by no smaller) for building TCP & UDP (inc multicast) based servers & clients ?
ie something that will just install quickly and cleanly by itself, with no external dependencies?
That should be pretty tiny and still useful.
Most of Kamaelia's components have no dependency on each other, so this would be fairly easy and quick to do. (It's perhaps better to think of Kamaelia as a toy box or tool chest or radio shack which you can just take the things out you want, rather than as a swiss army knife IMO) All the components depend on Axon, but then that's not too suprising - Axon is the component framework. But even that is small enough to be comprehensible.
Heck, I'll speculatively do it - it'll be quick to do and if you want to use it, feel free. :-)
Tell you what - how about I just package up the network components (no protocols, no pygame, no open gl, etc), along with a minimal set of other useful components as well by itself for you? ie the absolute minimum useful (by no smaller) for building TCP & UDP (inc multicast) based servers & clients ?
ie something that will just install quickly and cleanly by itself, with no external dependencies?
That should be pretty tiny and still useful.
Most of Kamaelia's components have no dependency on each other, so this would be fairly easy and quick to do. (It's perhaps better to think of Kamaelia as a toy box or tool chest or radio shack which you can just take the things out you want, rather than as a swiss army knife IMO) All the components depend on Axon, but then that's not too suprising - Axon is the component framework. But even that is small enough to be comprehensible.
Heck, I'll speculatively do it - it'll be quick to do and if you want to use it, feel free. :-)
9 months ago
in jessenoller.com » Blog Archive » What do you want to see in a concurrency talk… on jessenoller.com comments
Hmm. sorry for all the typoes - that'll teach me for posting responses to things when it's late! Anyhow, good luck :)
9 months ago
in jessenoller.com » Blog Archive » What do you want to see in a concurrency talk… on jessenoller.com comments
Jesse - please take a look at my Kamaelia talks on slideshare ...
http://www.slideshare.net/kamaelian/slideshows
... they're (literally) littered with numerous example systems which are highly concurrent, written in python and tested. Kamaelia is designed specifically with novices in mind, and the following 2 talks were given at pycon UK last weekend:
http://www.slideshare.net/kamaelian/practical-c...
http://www.slideshare.net/kamaelian/sharing-dat...
The latter one needs a bit of work based on feedback in the session, but the talks seemed highly accessible. (Indeed, kamaelia has been designed with accessibility in mind as a key design feature - it's also the driving long term reason behind Kamaelia's existence :-)
Kamaelia is primarily a message passing based system (generators, threads, processes as units of concurrency), but also includes a software transactional memory implementation for the times when you really do need to share data. (kamaelia's real underlying design is "no unconstrained data" and "if you have a piece of data and haven't given it back/to anyone else, your changes should be safe".
We're actually in the process of a revamp of the website.
If you want a selection of applications to talk about - there's examples using network servers, interactive pygame based applications, digital TV/PVR systems, offline batch transcoders, to name a few... I've noticed a tendency for people to relate to very concrete ideas - which is why this time round I chose a simple and direct game as the initial example.
The key thing which is perhaps different is that the focus in kamaelia is on making communicating systems useful through a component metaphor, rather than on "lets use concurrency for concurrency sake". ie use a pattern for software construction that happens to (deliberately :-) drop out as naturally highly concurrent.
Perhaps my most fun hack (10 hours or so) was the Speak And Write app - which uses handwriting recognition and speech synthesis to teach a child to read and write.
http://edit.kamaelia.org/SpeakAndWrite
Regarding the map-reduce idea - I'd personally love to see such a beast written using Kamaelia and would also be more than happy to see it merged onto /trunk ...
Perhaps the only downside of Kamaelia vs your talk is the choice of multiprocessing module in python 2.6 doesn't match the one we've been using in Kamaelia. (at some point we'll flip when we've got a chance - it'll make little difference to user code beyond being 1 less thing to install :)
Anyway, you did ask "what do you want to see ..." :-)
What I would hope is that you teach people the one rule that causes all problems in concurrent systems though - unconstrained shared data. If you make sure you don't have that, life becomes easy, fun and concurrent. If you do have any though, life often becomes very hard, and painful to debug.
Anyway, bets of luck for you talk - I hope it goes well :-)
http://www.slideshare.net/kamaelian/slideshows
... they're (literally) littered with numerous example systems which are highly concurrent, written in python and tested. Kamaelia is designed specifically with novices in mind, and the following 2 talks were given at pycon UK last weekend:
http://www.slideshare.net/kamaelian/practical-c...
http://www.slideshare.net/kamaelian/sharing-dat...
The latter one needs a bit of work based on feedback in the session, but the talks seemed highly accessible. (Indeed, kamaelia has been designed with accessibility in mind as a key design feature - it's also the driving long term reason behind Kamaelia's existence :-)
Kamaelia is primarily a message passing based system (generators, threads, processes as units of concurrency), but also includes a software transactional memory implementation for the times when you really do need to share data. (kamaelia's real underlying design is "no unconstrained data" and "if you have a piece of data and haven't given it back/to anyone else, your changes should be safe".
We're actually in the process of a revamp of the website.
If you want a selection of applications to talk about - there's examples using network servers, interactive pygame based applications, digital TV/PVR systems, offline batch transcoders, to name a few... I've noticed a tendency for people to relate to very concrete ideas - which is why this time round I chose a simple and direct game as the initial example.
The key thing which is perhaps different is that the focus in kamaelia is on making communicating systems useful through a component metaphor, rather than on "lets use concurrency for concurrency sake". ie use a pattern for software construction that happens to (deliberately :-) drop out as naturally highly concurrent.
Perhaps my most fun hack (10 hours or so) was the Speak And Write app - which uses handwriting recognition and speech synthesis to teach a child to read and write.
http://edit.kamaelia.org/SpeakAndWrite
Regarding the map-reduce idea - I'd personally love to see such a beast written using Kamaelia and would also be more than happy to see it merged onto /trunk ...
Perhaps the only downside of Kamaelia vs your talk is the choice of multiprocessing module in python 2.6 doesn't match the one we've been using in Kamaelia. (at some point we'll flip when we've got a chance - it'll make little difference to user code beyond being 1 less thing to install :)
Anyway, you did ask "what do you want to see ..." :-)
What I would hope is that you teach people the one rule that causes all problems in concurrent systems though - unconstrained shared data. If you make sure you don't have that, life becomes easy, fun and concurrent. If you do have any though, life often becomes very hard, and painful to debug.
Anyway, bets of luck for you talk - I hope it goes well :-)
2 replies
jnoller
I've looked at Kamelia before - it definitely looks like another "swiss army knife" framework - ala twisted, etc. I'm going to mention all of the frameworks (kamelia included) but I want to focus on the more primitive aspects of concurrency (multicore, messages) and how to build something using standard python.
Of course, I need a good network layer, so maybe I'll poke at Kamelia's :)
Of course, I need a good network layer, so maybe I'll poke at Kamelia's :)
Michael Sparks
Hmm. sorry for all the typoes - that'll teach me for posting responses to things when it's late! Anyhow, good luck :)