Apart from the issue of translators set up by untrusted users, here is another problem described.
IRC, freenode, #hurd, 2012-02-17
(Preceded by the memory object model vs block-level cache discussion.)
<slpz> what should do Mach with a translator that doesn't clean pages in a
reasonable amount of time?
<slpz> (I'm talking about pages flushed to a pager by the pageout daemon)
<braunr> slpz: i don't know what it should do, but currently, it uses the
default pager
<slpz> braunr: I know, but I was thinking about an alternative, for the
case in which a translator in not behaving properly
<slpz> perhaps freeing the page, removing the map, and letting it die in a
segmentation fault could be an option
<braunr> slpz: what if the translator can't do it properly because of
system resource exhaustion ?
<braunr> (i.e. it doesn't have enough cpu time)
<slpz> braunr: that's the biggest question
<slpz> let's suppose that Mach selects a page, sends it to the pager for
cleaning it up, reinjects the page into the active queue, and later it
founds the page again and it's still dirty
<slpz> but it needs to free some pages because memory it's really, really
scarce
<slpz> Linux just sits there waiting for I/O completion for that page
(trusts its own drivers)
<slpz> but we could be dealing with rogue translator...
<braunr> yes
<braunr> we may need some sort of "authentication" mechanism for pagers
<braunr> so that "system pagers" are trusted and others not
<braunr> using something like the device master port but for pagers
<braunr> a special port passed to trusted pagers only
<slpz> hum... that could be used to workaround the untrusted translator
crossing problem while walking a directory tree
translators set up by untrusted users.
<slpz> but I think differentiating between trusted and untrusted
translators was rejected for philosophical reasons
<slpz> (but I'm not sure)
<mcsim> slpz: probably there should be something like oom killer?
<mcsim> braunr: even if translator is trusted it could have a bug which
make it ask more and more memory, so system have something to do with
it. Also, this way TCB is increased, so providing port for trusted
translators may hurt security.
<mcsim> I've read that Genode has "guarded allocators" which help resource
accounting by limiting of memory that could be used. Probably something
like this could be used in Hurd to limit translators.
<antrik> I don't remember how Viengoos deals with this :-(
<braunr> mcsim: the main feature lacking in mach is resource accounting :p
<slpz> mcsim: yes, I think there should be a Hurdish oom killer, paying
special attention to external pagers
<braunr> the oom killer selects untrusted processes by definition (since
pagers are in kernel)
<mcsim> slpz: and what is better: oom killer or resource accounting?
<mcsim> Under resource accounting I mean mechanism when process can't get
more resources than it is allowed.
<braunr> resource accounting of course
<braunr> but it's not just about that
<braunr> really, how does the kernel deal when a pager refuses to honor a
paging request ?
<braunr> whether it is buggy or malicious
<braunr> is it really possible to keep all pagers out of the TCB ?
<antrik> mcsim: we definitely want proper resource accounting in the long
run. the question is how to deal with the situation that resources are
reallocated to other tasks, so some pages have to be freed
<antrik> I really don't remember how Neal proposed to deal with this
<slpz> mcsim: Better: resource accounting (in which resources are accounted
to the user tasks which are requesting them, as in the Viengoos
model). Good enough an realistic: oom killer
<antrik> I'm not sure an OOM killer for non-system pagers is terribly
helpful. in typical use, the vast majority of paging is done by trusted
pagers...
<antrik> and without proper client resource accounting, there are enough
other ways a rogue/broken process can eat system resources -- I'm not
convinced that untrusted pagers have a major impact on the overall
situation
<mcsim> If pager can't free resources because of lack, for example, of cpu
time it's priority could be increased to give it second chance to free
the page. But if it doesn't manage to free resources it could be killed.
<antrik> I think the current approach with default pager taking over is
good enough for dealing with untrusted pagers. the real problem are even
trusted pager frequently failing to deal with the requests
<braunr> i agree with antrik
<braunr> and i'm opposed to an oom killer
<braunr> it's really not a proper fix for our problems
<braunr> mcsim: what if needs 3 attempts before succeeding ?
<braunr> +it
<braunr> and increasing priority without a good reason (e.g. no priority
inversion) leads to unfairness
<braunr> we don't want to deal with tricky problems like malicious pagers
using that to starve other tasks
<mcsim> braunr: this is just temporary decision (for example for half of
second of user time), to increase probability that task was killed not
because of it lacked resources.
<braunr> mcsim: tunables should only affect the efficiency of an operation,
not its success
IRC, freenode, #hurd, 2012-02-19
<antrik> neal: the specific question is how to ensure processes free memory
fast enough when their allocation becomes lower due to resource pressure
<neal> antrik: you can't really.
<neal> antrik: the memory manager can act on the application's behalf if
the application marks pages as discardable or pagable.
<neal> antrik: if the memory manager makes an upcall to the application to
free some memory and it doesn't, you have to penalize it.
<neal> antrik: You shouldn't the process like exokernel
<neal> antrik: It's the developers fault, not the user's
<neal> antrik: What you need are controls that ensure that the user stays
in control
<neal> ...shouldn't *kill* the process...
<antrik> neal: well, how can I penalize a process that eats to much
physical memory?
<neal> in the future, you don't give it as much slack memory
<antrik> marking as pagable means a system pager will push them to the swap
partition?
<antrik> ah, OK
<neal> yes
<neal> and you page it more aggressively, i.e., you don't give it a chance
to free memory
<neal> The situation is:
<neal> you have memory pressure
<neal> you choose a victim process and ask it to free memory
<neal> now, you need to wait
<neal> if you wait and it doesn't free memory, you give it bad karma
<neal> if you wait and it frees memory, you're good
<neal> but during that window, a bad process can delay recovery
<neal> so, in the future, we give bad processes less time
<neal> but, we still send a message asking it to free memory
<neal> we just hope it was a bug
<antrik> so the major difference to the approach we have in Mach is that
instead of just redeclaring some pages as anonymous memory that will be
paged to swap by the default pager eventually if the pager in question
fails to handle them properly, we wait some time for the process to free
(any) memory, and only then start paging out some of it's pages to swap
<neal> there's also discardable memory
<antrik> hm... there is some notion of "precious" pages in Mach... I wonder
whether that is also used to decide about discarding pages instead of
pushing them to swap?
<neal> antrik: A precious page is ro data that shouldn't be dropped
<antrik> ah
<antrik> but I guess that means non-precious RO data (such as a cache) can
be dropped without asking the pager, right?
<neal> yes
<antrik> I wonder whether such a karma system can be introduced in Mach as
well to deal with problematic pagers
IRC, freenode, #hurd, 2012-02-21
<neal> antrik: One of the main differences between Mach and Viengoos is
that in Mach servers are responsible for managing memory whereas in
Viengoos applications are primarily responsible for managing memory.