Conversation with #inferno at Mon Apr 11 14:37:57 2011 on powerman-asdf@irc.freenode.net (irc)
(15:06:46) mjl-: powerman-asdf: that is impressive, the speedup with the strings
(15:07:16) mjl-: going for just packing/unpacking once sounds good.  hopefully it won't cause trouble with keeping data structures consistent (if you plan to change them)
(15:07:45) mjl-: option c doesn't sound too bad either.  but perhaps utf-8 packing/unpacking might still take some time.
(15:07:48) powerman: it wont if I'll use chan of JValue
(15:08:56) powerman: I just feel wrong to define architecture based on things like json performance in cluster environment nowadays.
(15:09:22) mjl-: true, but if that's the way it is...
(15:09:34) mjl-: disappointing that perl can do it faster...
(15:09:44) mjl-: or is that implemented by a c module?
(15:10:05) powerman: Rewriting more-or-less core and simple limbo modules in C just to get performance boost also feels wrong.
(15:10:18) mjl-: you could write a c parser and put it in inferno with a new interface, one without pick adt
(15:10:37) mjl-: well, because it is a core lib, it makes some sense
(15:10:46) powerman: Yeah, pure perl slower than limbo without jit in 2 times.
(15:10:49) mjl-: if that's what you have to do to get the performance you need
(15:15:23) powerman: I have an idea to test Go's json performance, but it fails to compile on my system. :( I suppose this may happens because of "hardened" kernel/gcc, and I'm going to switch off GrSec/PaX and try again. Because excluding hardened my Gentoo is just usual linux system.
(15:30:10) powerman: > mjl-: you could write a c parser and put it in inferno with a new interface, one without pick adt
(15:30:10) powerman: and how this can be possible? emulate pick adt without pick adt with something like adt{kind:int;i:int;r:real;s:string;a:array…}?
(15:30:53) mjl-: yes...
(15:30:55) mjl-: it's ugly
(15:31:06) mjl-: and error prone.  but if it's fast...
(15:31:29) mjl-: perhaps you can do the real parsing in c, then have a limbo-only module that converts them to/from the current json interface
(15:32:01) powerman: yeah, that's what I was thinking of
(15:32:30) powerman: and why I begin to profile string concatenations etc.
(15:33:24) powerman: because this way most of current json module will not be changed, and only some trivial low level data processing will be rewritten in C
(15:34:52) mjl-: exactly
(15:34:53) powerman: but, at a glance, it looks like this way I can speedup json in 2, maybe 3 times at most
(15:34:59) powerman: while I need 10
(15:35:50) mjl-: hmm, will it give only 2 or 3 times speedup?
(15:35:57) powerman: I've tried to estimate this by replacing such operations with no-op in current limbo json module.
(15:36:08) mjl-: ah
(15:36:39) powerman: like quote()
(15:37:24) powerman: while generating json it's most used and most complex part
(15:40:28) powerman: json generated by text(), no bufio etc. involved. and text() contain two trivial loops for array and object plus quote()
(15:41:00) powerman: and I don't think rewriting these loops in C will make n times speedup
(15:41:43) powerman: (but, with such a things, it's usually hard to know real speedup effect until you write real code and benchmark it)
(15:45:57) The account has disconnected and you are no longer in this chat. You will be automatically rejoined in the chat when the account reconnects.