Conversation with #inferno at Sun Apr 24 01:29:05 2011 on powerman-asdf@irc.freenode.net (irc) (01:31:08) powerman-asdf left the room (quit: Ping timeout: 260 seconds). (03:20:37) powerman: mjl-: (back to /opt discussion) how about installing opt man pages to /man/2opt/? this should solve possible conflicts with modified standard libs. standard man support /man/*/, so it works. also this way we can automatically include collection of man pages for all /opt packages on any website with all inferno man pages like http://doc.cat-v.org/inferno/manual_pages/ or http://powerman.name/Inferno/man/ (03:24:11) powerman: something like this, executed from user's ~/profile, with loop over all /opt/*/*/man/*/ dirs: (03:24:11) powerman: mount -a {mntgen} /man (03:24:11) powerman: bind -a /opt/someone/somepkg1/man/2 /mnt/2opt (03:24:11) powerman: bind -a /opt/someone/somepkg2/man/2 /mnt/2opt (03:49:41) jibun left the room (quit: Ping timeout: 240 seconds). (03:54:02) powerman: for d in /opt/*/*/man/*/ {and {ftest -d $d} {bind -a $d /man/`{basename $d}^opt}} (04:03:58) powerman: ok, it looks like mounting {mntgen} to /man somehow breaks man. :( so we'll have to work around this with memfs. this one works: (04:03:58) powerman: memfs -a /man (04:03:58) powerman: for d in /opt/*/*/man/*/ {and {ftest -d $d} {p=`{basename $d}; or {ftest -d /man/$p^opt} {mkdir /man/$p^opt}; bind -a $d /man/$p^opt}} (04:52:27) powerman: mjl-: and one more idea about installing opt commands: how about using sh's functions instead of trying to bind them to /dis/? when one opt app need to call another, it's not a problem to hardcode '/opt/someone/somepkg/dis/someapp.dis' path in first app, we just don't wanna type it manually. so, sh functions may helps: (04:52:27) powerman: for app in /opt/*/*/dis/*.dis { cmd := `{basename $app .dis}; '{ fn '^$cmd^' { '^$app^' $* } }'; } (04:54:24) powerman: with similar loop for executable sh-scripts (04:55:48) powerman: this won't support apps in dis/wm/ subdir, of course (04:56:35) anth_x: that apps thing's going to put a *lot* of stuff in your environment. (04:56:57) anth_x: that doesn't seem like a win. just bind in the stuff you want. (04:57:40) anth_x: for man, i'd bind /opt/pkgname/man/1 /man/1 (and so on) and rebuild the man index. (04:58:01) anth_x: if it works like plan9's (i've not checked this part), you'll just get a message about needing the missing page if you don't have it bound in. (04:58:33) anth_x: that doesn't tell you which package it's supposed to come from, but seems like a reasonable first approximation, without any need for the above stuff. (05:01:19) powerman: binding to /man/Xopt allow to distinguish between core and opt packages, plus allow to use both standard and modified modules like json - man 2 json; man 2opt json; include "json.m"; include "opt/powerman/json/module/json.m"; (05:03:41) powerman: as for /dis vs fn - what's the problem with adding all opt executables to /env? env is part of namespace, just like binds. (05:03:53) anth_x: but not packages with conflicting names, so it doesn't really solve the problem. (05:07:05) anth_x: it just significantly ups the odds of conflict between an opt executable and a user-defined function. (05:09:37) powerman: anth_x: in general, we can't automatically solve conflicting names in *NIX because all executables expected to be in single directory. keeping /opt structure as /opt/who/pkg solve conflicts between different pkg authors, using include "opt/who/pkg/module/pkg.m" with PATH con in pkg.m set to "/opt/who/pkg/dis/lib/pkg.dis" solve conflicts between modules with cheap cost - one long path name in one include command, which is acceptable. (05:11:24) powerman: but there are also man pages and commands, which should be accessed from single place and so may conflict. accessing commands from other commands can be solved by using long path to command, just like include in modules. that's also acceptable because long path name should be hardcoded just one. (05:12:40) anth_x: oh, you want to add another layer to the tree? /opt/who/pkgname, rather than /opt/pkgname? (05:13:43) powerman: this layer was there all of time :) actually, no one stops you from using /opt/pkgname for your own packages, but I'd prefer /opt/powerman/pkgname for my own, to avoid possible conflicts (05:14:00) anth_x: i agree direct reference to the long-form path makes sense in lots of places. (05:14:53) anth_x: hrm. i'd rather /opt/whatever directly. i ought to be able to swap implementations without having to change hard-coded references. (05:16:48) powerman: yeah, that makes sense. but if you just bind, say, /opt/powerman/json/ to /opt/json/ and include "opt/json/module.json" - it will works anyway, because /opt/powerman/json/ is still there and all hardcoded paths works (05:17:40) anth_x: yeah, but then another layer of indirection. how many json modules are we likely to really have, implementing different interfaces? (05:18:14) powerman: in ideal world there will be just one json module :) (05:18:28) powerman: one only true json module (05:19:03) anth_x: if you did need a differentiator, i prefer sun's naming scheme: /opt/SUNWfoo. but i'd still treat that as as-needed. (05:20:33) powerman: anyway. /opt/who/pkg allow to avoid conflicts, binding compatible pkgs from /opt/who/pkg to /opt/pkg allow switching implementations on the fly (05:20:59) powerman: /opt/whopkg will work no worse than /opt/who/pkg, it's just a question of taste (05:21:22) powerman: and all these three won't conflict (05:21:41) powerman: but their man pages and executables may conflict (05:22:18) anth_x: but i think you should be able to assume things like /opt/*/man, &c. (05:23:04) powerman: man pages conflicts can be avoided by binding /opt/who/pkg/man/2/ to /man/who_2/ instead of /man/2/. looks unusual, but will works. (05:24:39) powerman: or even to /man/2/ if you don't bother about man page conflicts at all. that's not really important (except when you need to convert all man pages to html and put on website, separating opt and core man pages) (05:26:32) powerman: command is more complex. if we bind -a /opt/who/pkg/dis/ /dis/ we won't be able to replace core command by opt command. if we bind -b /opt/who/pkg/dis/ /dis/, we lose access to core command replaced by opt command, which is even worse. (05:26:43) anth_x: i have a (partly working) man2html that serves a tree of manuals. it was originally done so we could do plan9, inferno, and our own project as seperate libraries on the same server. (05:26:51) powerman: using sh fn solve this (05:26:54) anth_x: i need to get that polished up... (05:27:48) powerman: it will replace core command with opt command when you call it as 'cmd', but you still have access to core '/dis/cmd' and opt '/opt/who/pkg/dis/cmd' (05:28:17) powerman: and it will work only in command line and sh scripts, i.e. exactly where it needed (05:28:49) anth_x: i don't think "losing" access to the core command is bad. in fact, i think it's generally the point of the naming collisions. (05:30:27) anth_x: if you don't mean to replace ls, don't name your command ls. (05:34:19) powerman: I don't like losing access to anything, but you right. But using bind -b /opt/who/pkg/dis/ /dis/ may (and will!) result in losing access to complete directories like /dis/wm/ if opt package prove few wm/something command (05:34:46) powerman: and here we back to complex recursive binds (05:36:51) anth_x: in my opt packages, i have a /dis/cmd that i bind -b onto /dis. it makes the /opt/pkg/dis namespace not exactly parallel the /dis namespace, but solves that problem (and, coincidentally, parallels /appl) (05:40:15) powerman: yeah, that's good idea. but there are 39 dirs like /dis/some/thing/, and even if we exclude /dis/lib/, there are still 24 such dirs (05:40:44) powerman: 24 isn't looks like occasional one or two (05:42:10) powerman: /dis/svc/httpd/ is good candidate to be improved/extended/replaced (05:42:18) anth_x: but why's that matter? (05:43:07) powerman: your solution with using dis/cmd/ for opt packages will also support dis/wm/ but won't support dis/svc/httpd/ (05:43:38) powerman: i.e. opt packages will be able to add commands to /dis/ and /dis/*/ but not to /dis/*/*/ (05:43:42) anth_x: oh, like if i have /opt/pkg/dis/svc, losing /dis/svc/httpd? (05:43:47) powerman: yep (05:43:50) anth_x: that's true, i hadn't run into that. (05:46:09) anth_x: the namespace is just manually constructed. its a fair rule to tell package maintainers to simply not cause conflicts (bind at lower levels or more specifically), but you can't really expect to solve any conflicts between packages that way. (05:47:55) anth_x: i mean, it's not necessarily manually constructed, but the namespace file is manually created by the author. (05:50:31) powerman: okay. this is an improvement over previous idea, so I think we should go this way for now (with who/ part optional): (05:50:31) powerman: bind -b /opt/who/pkg/man/2/ /man/2/ (05:50:31) powerman: bind -b to /opt/who/pkg/dis/cmd/ /dis/ (05:50:31) powerman: bind -b /opt/who/pkg/dis/DIR-NON-cmd-or-lib/ /dis/DIR/ (05:51:37) powerman: as for separating man pages to generate html version of both core and opt pages - that's specific task which can be done by unmounting /man/X/ and binding them to /man/somethingX/ or taking opt man pages from /opt/ by full path (05:52:35) anth_x: yeah. one namespace view doesn't have to serve all purposes. (05:54:19) powerman: losing access to core command also can be solved by unmounting before running some code which need access to original command. keeping in mind replacement opt command expected to be compatible with original command this should be rare case. (05:55:05) anth_r: i do wish /dis was constructed like plan9's /bin (or that we had a /bin or /exec or whatever that /dis was mounted on) (05:56:59) powerman: what's the difference between inferno's /dis and plan9's /bin? (05:57:22) anth_x: in plan9, the on-disk /bin has nothing in it. it's entirely constructed. (05:57:39) powerman: and how initially-empty /dis helps us? (05:58:25) powerman: is plan9 has subdirs in /bin like /dis/svc/httpd/? (05:59:10) anth_x: i'd like to see a /bin (although part of me would take the opportunity to rename it /exec or something that didn't imply everything in it was binary) where we could bind in things from /dis and (the currently non-existent) /sh) (06:00:53) anth_x: i can't think of any multi-level subdirs in plan9. (06:02:29) powerman: adding /exec is not a problem at all, trivial patch for sh to add one more dir for lookup, and you can start using it. problem is with Inferno's idea to have subdirs in /dis/, which is quite unusual and doesn't compatible with simple binds (06:07:05) anth_x: one of the guys i worked with at the labs did an inferno distribution that re-worked lots of the namespace, including bits like that. it was a real improvement, but there were a *lot* of rough edges. lots of things expect things in certain places. (06:07:31) anth_x: the debugger, for example, became a real pain. /exec might not be a problem, but it did kinda burn me from the experience. (06:16:15) anth_x: it's really irritating how much worse inferno's namespace is than plan9's. it's clear they weren't thinking of it as much of a real working environment. (06:27:49) powerman: anth_x, mjl-: http://pastebin.ca/2049993 that's proposed code to bind opt packages (06:29:35) anth_x: i have a /lib/namespace that i run at startup with this in the bottom: (06:29:36) anth_x: # Include site-wide optional packages here. (06:29:37) anth_x: run /opt/s1utils/lib/ns.s1utils (06:29:38) anth_x: run /opt/minerva/lib/ns.minerva (06:29:56) anth_x: as an example, /opt/s1utils/lib/ns.s1utils is: (06:30:13) anth_r: bind -a /opt/s1utils/module /module (06:30:14) anth_r: bind -a /opt/s1utils/dis/cmd /dis (06:30:14) anth_r: bind -a /opt/s1utils/man/1 /man/1 (06:30:14) anth_r: bind -a /opt/s1utils/man/2 /man/2 (06:30:14) anth_r: bind -a /opt/s1utils/man/4 /man/4 (06:30:26) powerman: 2+ level dir depth in dis/ not supported, package processing order undefined, but everything else should be ok (06:31:02) powerman: yeah, but that script proposed as general solution for /opt, which should be good enough for 99.9% packages (06:31:59) powerman: to avoid adding such standard commands first by package developers to opt/s1utils/lib/ns.s1utils and next by package users to /lib/namespace (06:32:04) anth_x: i list them manually and am okay with that. i think there's a lot of site-dependent behavior that makes it worth doing manually. you might also have some on the system that you don't always want in place. (06:33:38) powerman: can you provide an example of such site-dependent behaviour? as you said before, if you install opt package which provide own dis/ls.dis, then you probably did that intentionally to override default ls :) (06:35:30) anth_x: i don't have a local example, but imagine a package which provides a replacement ls. it might be something that some users want, but isn't really appropriate everywhere. (06:35:54) anth_x: leave it in /opt, but don't stick it in the system-wide /lib/namespace, and let users bind it in if they want it. (06:36:17) anth_x: doing it manually also allows site admins to choose who wins any naming conflicts. (06:37:18) powerman: ah, I see. that's because you edit /lib/namespace and made changes system-wide. I place it always to ~/namespace or ~/profile. that's because inferno usually installed by root into /usr/inferno, but run by usual user, and he doesn't have rights to edit /lib/namespace. (06:38:12) powerman: also, opt dir is user-provided too (06:38:24) anth_x: hrm (06:38:33) powerman: it's /usr/$user/opt, and bind to /opt by /usr/$user/profile (06:38:41) powerman: global /opt is empty (06:39:17) anth_x: i want to be able to use /opt packages for system services (web server, &c). (06:39:38) powerman: but isn't these services run in emu run by usual user? (06:39:47) powerman: or you start inferno as root? (06:40:11) anth_x: i run them on plan9. there is no root. :-) (06:40:12) powerman: even if you start it as root, you can use /usr/root/profile to setup (06:40:20) anth_x: (actually, root's the name of my cpu server. :-) (06:40:57) powerman: i just prefer to keep inferno installation clean from modifications, and keep all modification in home dir (06:41:32) powerman: with only exception to opt C-modules, which has to be compiled into inferno kernel :( (06:43:30) powerman: as for resolving naming conflicts manually, it's easier to manually re- bind -b one-two packages again to /dis after all opt packages was bind by general script. hoping there are no rootkit in these opt packages which replace bind.dis :) (11:35:15) bvalek2 [5e152de0@gateway/web/freenode/ip.94.21.45.224] entered the room. (12:03:21) mjl-: powerman-asdf1: the opt bind script looks good (12:03:26) mjl-: i think that's convenient (12:03:34) mjl-: btw, i wouldn't mind using bind -a (12:04:10) mjl-: then the user has to manually add binds when it wants to override an existing program. which feels a bit safer, and should not happen so often that it is annoying. (12:04:37) mjl-: one thing i really like to get rid of is: paths to opt in the .m and/or makefiles (12:04:56) powerman: why? (12:05:14) mjl-: not entirely sure :) (12:05:22) mjl-: having the project names hardcoded feels a bit weird (12:05:31) mjl-: eg /opt/mjl/mymod (12:05:49) mjl-: mostly aesthetics i think... (12:06:07) powerman: it's not "project", it's module name. hardcoding "json.m" or "opt/powerman/json/module/json.m" is same, just a bit longer (12:06:42) powerman: this way you may have "util.m" and I too, and they will not conflict (12:06:55) powerman: also, it's nice to see vendors of used opt modules (12:07:12) powerman: to know where to search for this module sources, etc. (12:07:22) mjl-: myeah, those long paths just look really ugly to me :) (12:07:33) mjl-: but you're that it is clearer (12:07:48) powerman: and conflict-safe (12:09:51) mjl-: tre (12:09:53) mjl-: true* (12:10:58) mjl-: ok, so i should just start using it :) (12:11:41) mjl-: is the -I$ROOT and -I/ already in inferno-os? (12:11:50) mjl-: in the mkfiles (12:18:04) mjl-: powerman-asdf1: you install the opt packages in $home/opt, right? (12:18:38) mjl-: if you compile from the host os, does that work correctly? with $ROOT/opt. the $home/opt/* are not found then right? (12:20:42) mjl-: heh, i don't want to put .dis files in my repo's. but then i'll get the "empty dirs in hg" problem... (12:21:33) powerman: i've configured it to work both in host os in inferno (12:22:00) mjl-: how did you do that? (12:22:51) powerman: in host os I've INFERNO_HOME variable set to dir which should become inferno's home dir when starting emu: (12:22:51) powerman: INFERNO_HOME=/home/powerman/inferno (I prefer to separate host os home dir and inferno home dir) (12:23:35) powerman: next, path to opt-files set without starting slash, i.e.: include "opt/vendor/pkg/…"; (12:23:57) powerman: this way it become relative to limbo's -I options (12:24:41) powerman: next, in mkfile I use something like: (12:24:41) powerman: OPT_ROOT=$INFERNO_HOME (12:24:41) powerman: LIMBOFLAGS=$LIMBOFLAGS -I$OPT_ROOT (12:24:53) powerman: <$OPT_ROOT/opt/powerman/tap/mktest (12:26:22) powerman: that's needed only to locate files from other opt packages (12:26:29) powerman: like your util0.m (12:26:56) powerman: if opt module have no dependencies, it doesn't need to know about /opt at all, and all paths in mkfiles can be relative (12:27:59) powerman: that was in host os; now, inside inferno I've added this to ~/profile: (12:27:59) powerman: fn limbo { builtin limbo -I/ $* } (12:31:42) mjl-: ok, good (13:13:45) powerman: mjl-: just found one more thing probably related to this setup: in ~/.bashrc: (13:13:45) powerman: function mk() { /usr/inferno/Linux/386/bin/mk "$@" ROOT=/usr/inferno; } (13:32:09) bvalek2 left the room (quit: Ping timeout: 252 seconds). (13:35:10) jibun [~jibun@dsl-trebrasgw1-fff0c300-37.dhcp.inet.fi] entered the room. (14:09:34) bvalek2 [5e152de0@gateway/web/freenode/ip.94.21.45.224] entered the room. (14:57:53) jibun left the room (quit: Ping timeout: 260 seconds). (16:00:39) bvalek2 left the room (quit: Ping timeout: 252 seconds). (16:46:56) Fish- [~Fish@sat78-8-88-174-225-4.fbx.proxad.net] entered the room. (17:16:55) Fish- left the room (quit: Remote host closed the connection). (17:38:16) bvalek2 [5e152da5@gateway/web/freenode/ip.94.21.45.165] entered the room. (18:33:04) jibun [~jibun@dsl-trebrasgw1-fff0c300-37.dhcp.inet.fi] entered the room. (19:08:45) bvalek2 left the room (quit: Ping timeout: 252 seconds). (19:11:53) The account has disconnected and you are no longer in this chat. You will be automatically rejoined in the chat when the account reconnects.