Kevin
Is this you? Claim Profile »
6 months ago
in Cousin Mikey is taking an interest in Twitter (Scripting News) on Scripting News
Yes... employ aggressive caching. Distributed caching so that it can be shared with multiple instances of web-app server.
6 months ago
in Cousin Mikey is taking an interest in Twitter (Scripting News) on Scripting News
WebLogic (BEA), Tibco and webMethods are three companies that made a killing with enterprise messaging product before 2000. Later IBM came out with MQ Series. Microsoft also has a messaging product. Of course, Oracle - lately acquired BEA. Except for Microsoft, all these products use Java Messaging System (JMS). Today, there is an open messaging standard in Java 2 Enterprise Edition (J2EE) -- EJB (Enterprise Java Bean) MDB (Message Driven Bean).
Today, you can use EJB MDB for free with JBoss and Sun GlassFish. If you want to implement messaging in the Web Tier you can use JMS directly in Tomcat. JMS and MDB are very stable standard products. They are very easy to use. The difficulty, as Sirram Krishnan pointed out in http://www.sriramkrishnan.com/blog/2008/05/what... -- it's difficult to designed distributed messaging systems. There is no distributed programming tools.
Personally, I am very interested in creating an enterprise version of Twitter. One that can be deployed as a node. The nodes will talk to each other via JMS. Users one node can follow users in another node. When a node is down, we can still get the feeds from backup silos. Silos are nodes. Feeds and filters are handle by other nodes. This can be done in Java.
Today, you can use EJB MDB for free with JBoss and Sun GlassFish. If you want to implement messaging in the Web Tier you can use JMS directly in Tomcat. JMS and MDB are very stable standard products. They are very easy to use. The difficulty, as Sirram Krishnan pointed out in http://www.sriramkrishnan.com/blog/2008/05/what... -- it's difficult to designed distributed messaging systems. There is no distributed programming tools.
Personally, I am very interested in creating an enterprise version of Twitter. One that can be deployed as a node. The nodes will talk to each other via JMS. Users one node can follow users in another node. When a node is down, we can still get the feeds from backup silos. Silos are nodes. Feeds and filters are handle by other nodes. This can be done in Java.
1 reply
knackeredhack
Much as I enjoy the open Twitter, I do think that an effective enterprise application is an enormously exciting prospect in the hands of good management, or where open collaboration is already embedded within the organization. Perhaps they are the same thing. I suspect that most companies will be put off by the way Twitter is ordinarily described, and no doubt the security industry won't be long in calculating how much staff time has been "wasted" on Twitter.
6 months ago
in Cousin Mikey is taking an interest in Twitter (Scripting News) on Scripting News
Twitter team needs to look towards NASDAQ or New York Stock Exchange. The # of people following a channel is the same as the number of institutions following a ticker in Wall Street. Twitter is less complex than Wall Street because Twitter only displays the last posting from the people you are following.
Wall Street is very distributed using message queues to distribute trades to multiple nodes. Everyone does not get real-time updates. You pay to get real-time update. Wall Street has different tiers of data services, majority of the people are getting delayed quotes. Frankly, I don't see why Twitter need real-time for everyone.
Twitter must first be refactored into a distributed systems. It must explore caching to reduce the # of DB hits. Twitter already admitted that they have a CMS-like architecture. First, they need to rethink this. Then, they need to distribute workload to different servers using async. events (MQueue or Message Driven Beans). The Jabber interoperability, for example, can be handled by another server.
Kevin
Wall Street is very distributed using message queues to distribute trades to multiple nodes. Everyone does not get real-time updates. You pay to get real-time update. Wall Street has different tiers of data services, majority of the people are getting delayed quotes. Frankly, I don't see why Twitter need real-time for everyone.
Twitter must first be refactored into a distributed systems. It must explore caching to reduce the # of DB hits. Twitter already admitted that they have a CMS-like architecture. First, they need to rethink this. Then, they need to distribute workload to different servers using async. events (MQueue or Message Driven Beans). The Jabber interoperability, for example, can be handled by another server.
Kevin
1 reply
Jim McCusker
@Kevin Leong
Twitter doesn't display just the last message from people you follow. It displays everything. It's not uncommon for me to see multiple messages from the same person in my twitter stream of people I follow.
Using a distributed MQ-like infrastructure makes a lot of sense, but it also needs to have a heavy dose of load balancing.
Twitter doesn't display just the last message from people you follow. It displays everything. It's not uncommon for me to see multiple messages from the same person in my twitter stream of people I follow.
Using a distributed MQ-like infrastructure makes a lot of sense, but it also needs to have a heavy dose of load balancing.
9 months ago
in Guidelines for competing with Twitter (Scripting News) on Scripting News
Here are some of my thoughts
+ TwitterNode with Federation of TwitterNodes is a really cool idea.
+ www.twitter.scripting.com is a node and www.twitter.scripting.com/dave a channel
+ Node can communicate with www.twitter.com
+ Node can be service by a redundant backup node if the real node is down – thus reliability
+ For auditability, each node will archive in/out feeds
+ User should be able to move from one node to another. Cannot guarantee he/she will have the same channel name, but should be able to move archives to new node.
+ Each node must manage sign-on
+ To support payload (video, audio, photo – which Dave has been playing with) each node will have to handle media transcoding
+ Not sure XMPP is needed between nodes or users to node. SIP connection negotiation is not cheap. Real-time notification can be done with reverse AJAX.
+ Biz-model (follow to mySQL $1B or JBoss) -- advanced features for ISP, carriers and corporations.
+ TwitterNode with Federation of TwitterNodes is a really cool idea.
+ www.twitter.scripting.com is a node and www.twitter.scripting.com/dave a channel
+ Node can communicate with www.twitter.com
+ Node can be service by a redundant backup node if the real node is down – thus reliability
+ For auditability, each node will archive in/out feeds
+ User should be able to move from one node to another. Cannot guarantee he/she will have the same channel name, but should be able to move archives to new node.
+ Each node must manage sign-on
+ To support payload (video, audio, photo – which Dave has been playing with) each node will have to handle media transcoding
+ Not sure XMPP is needed between nodes or users to node. SIP connection negotiation is not cheap. Real-time notification can be done with reverse AJAX.
+ Biz-model (follow to mySQL $1B or JBoss) -- advanced features for ISP, carriers and corporations.