Conversation with #inferno at Sat Apr  9 20:00:19 2011 on powerman-asdf@irc.freenode.net (irc)
(20:30:51) bugQ [~bug@c-67-171-127-76.hsd1.ut.comcast.net] entered the room.
(22:38:36) bugQ left the room (quit: Quit: 1 sec, trying to find today's log).
(22:39:32) bugQ [~bug@c-67-171-127-76.hsd1.ut.comcast.net] entered the room.
(00:31:44) tensai_cirno [~cirno@77.232.15.216] entered the room.
(00:37:11) Fish left the room (quit: Quit: So Long, and Thanks for All the Fish).
(04:35:38) bugQ left the room (quit: Ping timeout: 248 seconds).
(04:42:40) bugQ [~bug@c-67-171-127-76.hsd1.ut.comcast.net] entered the room.
(08:15:55) bugQ left the room (quit: Remote host closed the connection).
(09:06:50) perdiy [~mkhl@sxemacs/devel/perdix] entered the room.
(09:08:16) perdix left the room (quit: Ping timeout: 246 seconds).
(13:57:14) KBme left the room (quit: Excess Flood).
(13:58:15) KBme [~KBme@2001:470:1f13:94::3] entered the room.
(14:31:42) powerman: I've performance issue with JSON. Current data flow in our system is about 250_000 items per minute, each item packed into JSON took about 300 bytes. We've a lot of network services which works like a pipe (checking, normalizing, filtering, etc. these items). So, each service got json as input, unpack it, process, pack back to json and sent to it output.
(14:32:46) powerman: Some services are in Perl, some in Limbo. Perl use JSON::XS module, which is implemented in C, and work ~10 times faster than Limbo's json module.
(14:34:19) powerman: And that's with jit. Without jit Limbo become ~7 times more slower (to be honest, pure perl's JSON::PP is even 2 times slower than Limbo without jit).
(14:38:07) powerman: Now, we have to develop complex and flexible data filtering service, which I plan to implement in Limbo, using ~10-15 independent processes/modules, which will do parallel data analyse with different algorithms, produce some "expert knowledge" (of different types) as result of this analyse, keep these "knowledge" in databases (one per each type of "knowledge"), and use these databases to filter data items.
(14:38:10) The account has disconnected and you are no longer in this chat. You will be automatically rejoined in the chat when the account reconnects.