Avoiding CPU Speed Scaling – Running CPU At Full Speed

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power.

Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on lower CPU speed, then they could. This can be easily checked with this command:

#grep -E '^model name|^cpu MHz' /proc/cpuinfo

What you can see:

model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz

cpu MHz : 1600.000

...

or

model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

cpu MHz : 1596.000

...

or even

model name : Intel(R) Core(TM)2 CPU 4400 @ 2.00GHz

cpu MHz : 1000.000

model name : Intel(R) Core(TM)2 CPU 4400 @ 2.00GHz

cpu MHz : 600.000

Oops, we are paying for 2 Core 2Ghz CPU that runs on 600Mhz on one core and 1000Mhz on another!!!

There will be other lines for all the CPUs/cores/threads, probably with the same values.

This feature is nice, if we are running workstation, but what I've noticed, we do have the same CPU throttling on Ubuntu Server 10.04 builds and on CentOS 5.3, 5.4 and 5.5 builds (thus on RedHat too).

After hours of digging google, I've found that:
- this problem is very common
- there are several bug reports about this issue
- this is not BIOS settings problem, because on dual boot systems, CPU runs at full speed on Windows
- there are no 100% working solutions or they are too difficult to find
- this is not a bug, but a 'feature' of the new kernels, it is implemented differently on 2.6.18 (CentOS) and 2.6.32 (Ubuntu).

Here is a tip how to disable it on running system:

1) Check that 'kondemand' thread is running, run as root:  "pgrep -lf ondemand"

the output should be like:

# uname -a

Linux boston07 2.6.18-164.6.1.el5 #1 SMP Tue Nov 3 16:18:27 EST 2009 i686 i686 i386 GNU/Linux

# pgrep -lf ondemand

1444 kondemand/0

1445 kondemand/1

2) Check that current cpu speed differs from the maximum:

# grep -E '^model name|^cpu MHz' /proc/cpuinfo

model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

cpu MHz : 1596.000

model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

cpu MHz : 1596.000

3) Change CPU governor from 'ondemand' to 'performance' for all CPUs/cores, run as root:

for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance > $CPUFREQ; done

4) Check that your changes have been applied:

# grep -E '^model name|^cpu MHz' /proc/cpuinfo

model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

cpu MHz : 2394.000

model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

cpu MHz : 2394.000

5) If you are running 'cpuspeed', 'cpufreqd', 'powerd' or other daemons, that can control CPU stepping, just stop them, if you really need to run your system on 100% of the CPU speed.
On CentOS:

# service cpuspeed stop

6) On Linux 2.6.32 (On RedHat 6, and Oracle Unbreakable Linux 6) remove CPU scaling kernel modules:

# lsmod | grep ondemand

cpufreq_ondemand 8764 0

freq_table 3751 2 cpufreq_ondemand,acpi_cpufreq

# rmmod cpufreq_ondemand acpi_cpufreq freq_table

Ensure that no 'kondemand' kernel threads are running:

# pgrep -lf kondemand

#

7) To make changes permanent (on reboot):

- On Ubuntu, modify /etc/init.d/ondemand script:

change this

echo -n ondemand > $CPUFREQ

to this:

echo -n performance > $CPUFREQ

OR ALTERNATIVELY just remove all references to ondemand from /etc/rc?.d/

rm -f /etc/rc?.d/S99ondemand

- On CentOS, just create a new script /etc/init.d/ondemand:

#! /bin/bash

#

# ondemand sets cpu govermor

#

# chkconfig: 2345 10 90

#

# description: Set the CPU Frequency Scaling governor to "performance"

#

### BEGIN INIT INFO

# Provides: $ondemand

### END INIT INFO

PATH=/sbin:/usr/sbin:/bin:/usr/bin

case "$1" in

start)

for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

do

[ -f $CPUFREQ ] || continue

echo -n performance > $CPUFREQ

done

;;

restart|reload|force-reload)

echo "Error: argument '$1' not supported" >&2

exit 3

;;

stop)

;;

*)

echo "Usage: $0 start|stop" >&2

exit 3

;;

esac

then enable it:

chmod +x /etc/init.d/ondemand

chkconfig --add ondemand

service ondemand restart

I'm using 'ondemand' name of the script, this may be a little bit misleading (because really it is a 'performance'), but you may change it.

Reference here: http://www.servernoobs.com/avoiding-cpu-speed-scaling-in-modern-linux-distributions-running-cpu-at-full-speed-tips/?doing_wp_cron=1390110978.2860989570617675781250#

5 Comments

  • Mac says:

    Great write up, I am looking to buy a new workstation for home for my primary CentOS6 box. I am going to make sure the cores are at 100%, this would be helpful in studying for certifications where you are running a lot of vm's.

  • I have some pretty serious reservations about this article. You should make note that following this procedure eliminates on-demand clocking, so your CPU is always running at the top speed bin, and that this will have an impact on electrical usage, sometimes dramatically so.

    More importantly, though, on modern CPUs with Turbo Boost features, this will disable them. For example, the i7-4800MQ in my laptop idles down to 800MHz, and runs at 2.7GHz full load. However, if only one core is being heavily loaded, it can clock up to 3.7GHz instead. If I use your "fix", the CPU will stay locked at 2.7GHz, since the other cores will never clock down at idle.

Leave a Reply

XHTML: You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

%d bloggers like this: