1. the problem?

Linux does a good job making sure that all processes are treated fairly...but...the problem is that not all processes need fair treatment. Some need more than fair, and some don't mind less.

2. example - root process

X Windows just doesn't work well with modern Linux (kernels 2.6.x and before) unless it gets a big priority boost. So, many distributions (e.g Debian) now come with a nice -10 boost hard-wired into the X start scripts.

3. another example - user process

Another example is the sound daemon artsd. Unlike X Windows, the process runs under the users account. Unless this process is run as a realtime process, the sound will get choppy when using the computer. So, there's a SUID-root wrapper artswrapper which is needed to allow the user to run under elevated priority.

4. general problem?

The general and growing problem is that many processes require elevated priority - each of the programs for video, audio, VoIP, CD/DVD burning - so each of them, in theory, should have SUID-root wrappers setting appropriate boosts!

There's also the converse problem - background processes affecting the performance of everything else. Nobody is waiting for these processes, so they should be run with lowered priorities, solving the system performance.

There's in fact a whole spectrum of processes in between these two extremes of realtime to background. It is important to ensure that, as far as possible, processes preempt each other only according to where they lie on the spectrum.

Instead of adding hard-wired priority tweaks in scripts and SUID wrappers, and then trying to harmonise those settings....

5. general solution

niced is designed to be a single general solution across servers, desktops, and laptops. It supports all machines types - single processor, hyper-threaded, dual-core, dual processor, SMP, clusters. niced unmodified is a better default than an unmodified system. niced is easily customised to your specific needs. This is a dynamic and centralised solution to the problem of prioritising these exceptional process in a nice daemon - niced.

6. what does it do?

It changes the priorities of running processes (using nice levels, realtime levels, and realtime scheduling policies) based on process name, use and load. It reruns upon load change, upon screensaver changes, or periodically.

7. where to get it?

Download latest version 0.7.4 now. This is the old production release (do not download - this version is now outdated).

Download previous version 0.6 . The previous production version (also outdated).

The development version 0.8a release is available too (also outdated). Note this version is for testing purposes only and not for production use.

The current development version 0.8a pre-release (this is now the only up-to-date version - get this version if in doubt which to get). Note this version may be completely untested or even the same as above, is for testing purposes only and not for production use.

8. how to install it?

After download:

mv niced.sh /usr/local/bin/niced

chmod +x /usr/local/bin/niced

9. how to run it?

As root

niced &

10. how to run it and see what it's doing?

As root 

niced -v &

11. ...even more details?

As root 

niced -vv &

12. get help?

niced -help

13. get version?

niced -version

14. what overhead does running niced create?

niced code has been optimised to create negligible overhead, and runs at lowest priority.

15. I like it - what about running it all the time?

niced can be run all the time on all machines. The much improved overall performance far outweighs the small overhead (FAQ 14.) involved in running niced.

16. How to run niced from boot?

Just add this to your boot scripts

To install this file on Debian:

mv niced-boot.sh /etc/init.d/niced

chmod +x /etc/init.d/niced

and then can test with a manual start/stop with:

/etc/init.d/niced start

and then you will need to enable it at boot. For more info on how to do that with your distribution - see your distributions documentation.

17. free lunch?

You don't make my processor faster - how can things run faster?

The long answer is to read the niced theory. The short answer is that things can run faster, but normally only at the expense of other things running slower. There are overall gains to be had too due to increased parallelism or less context switches/cache flushes.

18. kernel versions?

Realtime support has been included in the kernel for a long time.

SMP nice rebalancing requires a future kernel >= 2.6.18, which balances the run queues based on priority. (You should check on the exact status of SMP nice rebalancing in the kernel).

Batch scheduling has been implemented (kernel >= 2.6.16), but currently niced does not take advantage of that. Support will be added in future, but I currently do not know of any standard processes that would benefit.

ionice requires kernel >= 2.6.13 .

19. required software?

For realtime support, /usr/bin/chrt is needed. e.g. Debian users can install chrt with

apt-get install schedutils  and alternatively in future apt-get install util-linux  (starting with util-linux version 2.13-pre1).

For default ionice support, no extra software is needed, but for advanced ionice support /usr/bin/ionice is needed. e.g. Debian users can install ionice with

apt-get install util-linux 

(starting with util-linux version 2.13-pre1), or alternatively also apt-get install schedutils (version >= 1.5.0). Currently, debian only has util-linux version 2.12, so until update, you can get the files directly from ftp://ftp.kernel.org/pub/linux/utils/util-linux/ .

20. is niced a good idea?

Yes. Biggest benefits are for desktop Linux, but for all machines at all times, niced helps prevent choppy VoIP, sound, video, X, running of multimedia programs, higher general interactive performance, and higher throughput. Most importantly, it is well tested, configurable and has minimal overhead.

21. do I have to configure anything?

No, you don't have to, but you may have to add a program or two if they are not listed.

22. isn't this all a bit annoying to have to list these process names?

Yes, but I've done that for you! niced centralises this information that otherwise would have to be sprinkled about the system. Centralising also allows further optimisations like dynamically changing things.

23. what language is niced written in?

niced is currently written in shell script (also see FAQ 14, 24, 25.).

24. why is niced written in shell script?

niced is a fully functional prototype to evaluate the benefits of the method, and all the priority tweaks. As a prototype, shell script is quick to change and customise and has minimal memory requirements. (also see FAQ 25.).

25. what language will niced 2.0 be written in?

The plan is to rewrite niced in C or C++.

26. are there any security issues?

Do not use on any machine that is used by an untrustworthy user. Any user can create processes that starve other users and eventually crash the machine. If you know what you are doing, feel free to modify niced to restrict it to certain trustworthy users.

27. can I use niced if I already manage priorities (using nice/renice/chrt)?

Either you manage everything yourself manually, or you let niced automate this. If you try to mix the two you are likely end up with something undesirable. If you already manage your nice levels, realtime levels and scheduling policies for everything that matters on each machine, then you probably do not need niced. Also see (FAQ 26).

28. can I use niced with a batch queuing system?

yes. Just take note of (FAQ 26, 27).

29. what to do if a high priority process is hanging and using all the CPU?

If you have this sort of problem, you should always run niced with the safe option

as root 

niced -safe &

you can also login now (or later) on tty2. To do that from X-Windows

Control-Alt-F2

login as root (or the user who's process causes the problem)

you're now ready for when the process hangs....

When it does hang,

Control-Alt-F2

and kill that process

This works as niced -safe sets processes using tty2 to be nice -20 (and RT 99) to allow you to do emergency work by logging in there. Note that it's best to login there before you know you need to use it. (TODO the following might not be needed) If you wait to login until after the process is hanging, then you will have to wait for the next periodic run of niced to renice your process AND you will have to wait until (or if) the hanging process lets niced have come CPU time to run. If you actually use tty2, then set this to another tty.

30. niced made my laptop catch fire!?

This software comes with no warranty whatsoever. Total data and hardware loss can result from use of this software. You might find it does none of those too.

31. Licence?

GPL

32. version history

(see FAQ 13)

version 0.8a 2007.05.30
              - -update (testing on Kubuntu 7.04)
version 0.7.4 2006.07.27
              - -cput cpu throttle in idle, -ncput no cpu throttle (default)
              - more screensavers supported
              - screensaver mode bugfixes
              - priority tweaks
version 0.7.3 2006.06.28
              - allow detecting screensaver turning off at all times
              - bu, -nbu buffer underrun, no buffer underrun (default)
              - prevent load thrashing (3x minimum loops due to constant load change)
              - b, -nb boost option for apps (default off) (poor mans focussed boost)
              - help and version options
              - better error messages
              - safe sets max RT priority 99
              - RT priority tweaks
              - bugfix chrt only takes 1 PID
              - bugfix some RT levels not set
              - priority tweaks
              - timing tweaks
version 0.7.2 2006.05.22
               - screensaver mode weights wait times
               - screensaver mode recognises xlogin -> (level 1 of screensaver)
                   - screensaver going off, but xlogin still on -> screensaver off (for 1 loop)
               - screensaver automatic minor mode (2 levels) (default)
               - safe supports realtime
               - nosafe option
               - priority modes: automatic (default), desktop, server
                   - server mode now distinct
                   - automatic defaults to desktop unless running server trigger processes
               - RT on/off command line flags
               - internal changes
               - pruning all empty levels for loop speed
               - loop optimised
               - syntax cleanup
               - priority tweaks
               - timing tweaks
version 0.7.1 2006.04.21
               - bugfix
               - load based min and max loop period
               - timing tweaks
version 0.7 2006.04.20
               - realtime support
               - load change run triggering
               - load based wake frequency
               - loop optimised
               - priority modes: server (default), desktop
                   - server currently only changes a couple of processes: squid and httpd
               - internal changes
                   - misc changes
                   - rearrange variables and code ready for modes
               - server mode currently best default
               - priority tweaks
 version 0.6 2006.04.11
               - safe mode
               - load based run frequency
               - priority tweaks
 version 0.5 2006.04.07
               - first public version
               - priority tweaks
 version 0.4 2006.03.07
               - loop
               - priority tweaks
 version 0.3 2006.01.20
               - laptop mode (preliminary)
               - priority tweaks
 version 0.2 2006.01.17
               - diagnostics
               - priority tweaks
 version 0.1 2006.01.16
               - first working version

33. Future?

A complete list of TODOs are available:

niced -todo

The main points are: 

Version 0.8 supports Linux distributions 2007, safe, correct multimedia support, thread support (all previous versions obsolete without this)

Version 0.9 support non KDE WM, batch scheduling, rewrite

Version 1.0 revisit screensaver mode

Version 2.0 a complete rewrite in C is planned. (FAQ 25.)

34. Similar Projects

VeryNice - does nice up and down

AND - Auto Nice Daemon - does not nice up, so cannot be used like niced. Great system for renice-ing long jobs.

35. niced Homepage?

http://jrweare.googlepages.com/niced.html

36. more information?

For more information, see the niced theory.

37. Feedback?

Mail me

 

 

Johnathon Weare. © 2007 Johnny Weare