From: Pete Zaitcev <zaitcev@redhat.com>
To: iwhd-devel@lists.fedorahosted.org
Cc: Project Hail List <hail-devel@vger.kernel.org>
Subject: Autostart challenges
Date: Mon, 4 Oct 2010 18:15:55 -0600 [thread overview]
Message-ID: <20101004181555.550dc5b7@lembas.zaitcev.lan> (raw)
After thinking about autostart for a bit (a week or two), my primary
concerns about it are two:
1. what is it for
2. what are its failure modes
1: I agreed to implementing autostart largely because Jim had trouble
configuring tabled for iwhd and I took it as evidence that there was
a space for simplification and/or automation there. However, let's
look it from the perspective of a Deltacloud hacker who just wants iwhd
for testing. Surely they would be served with the filesystem back-end?
What I missed is that running tabled is only interesting when you debug
iwhd's S3 backend, hack on iwhd's fail-over, or want to measure some
performance in a real private cloud. Unfortunately, autostart cannot
do any of that (even if it knew how to do it, it would require assumptions
about the minimal cloud, and then it becomes not very auto). And for
the Deltacloud hacker's testing a simple iwhd-1node-example.json
would be sufficient.
2: IMHO the fully automated and under-the-hood operation is only
valuable if it's completely bulletproof. Unfortunately, it's a challenge.
What if something fails to start, fails to stop, or otherwise breaks.
The patch that I posted deals with restarts by assuming that if
something is running and processes requests, it is a healthy service.
Thus it does not need to kill anything ever, in theory. In practice,
however, a partial failure is possible that requires the user to
kill processes, remove some some directories, or both. So to an
extent the promise of autostart is a lie. I'm not comfortable with it.
Now, a few specific comments on this patch:
http://marc.info/?l=hail-devel&m=128622349329834&w=2
#1: We use an explicit -a switch to launch autostart (plan the
same on iwhd too). This is done to awoid confusing printouts and
spawning anything if a user runs the program without arguments
or with -h.
Same logic applies to -c and -a together.
#2: There is no kill handler (iwhd has none, tabled does DB shutdown
on signal). It's mostly for the concern of not being reliable.
#3: The credentials for accessing services are not established yet.
Everything works without due to implementation deficiencies.
#4: Current directory is used for all the back-end data (tabled
inherits it from iwhd, which is nicely automagic and needs no
configuration). One warning though - do not cd /tmp.
#5: There are a couple of things that need fixing in Hail:
- We do not delete the state that Hail (e.g. CLD and tabled)
accumulates, so CLD session locks persist across restarts.
The cld has to be fixed to clear obsolete locks right away
instead of letting them exire in a minute.
- All the constant verbosity that tabled produces even when idle
is rather annoying, even after being redirected to a log file.
It must go.
BTW, the patch itself offers one potential advantage to tabled:
removal of magic delays from "make check" scripts. So we may yet
to get it committed, but even so I am questioning if iwhd needs
the autostart.
-- Pete
reply other threads:[~2010-10-05 0:15 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101004181555.550dc5b7@lembas.zaitcev.lan \
--to=zaitcev@redhat.com \
--cc=hail-devel@vger.kernel.org \
--cc=iwhd-devel@lists.fedorahosted.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).