2012年11月9日星期五

Job 相同 Unique Key 的处理

测试 1
<?php

$start_time = date('Y-m-d H:i:s');
echo "Start: " . $start_time . "\n";

$gmc= new GearmanClient();
$gmc->addServer();
$gmc->setStatusCallback("reverse_status");
$gmc->setCompleteCallback("reverse_complete");
$gmc->addTask("reverse", "Hi! - $start_time", null, 1);


if (! $gmc->runTasks()) {
    echo "ERROR " . $gmc->error() . "\n";
    exit;
}


function reverse_status($task) {

    echo "STATUS: " . $task->unique() . ", " .
    $task->jobHandle() . " - " .
    $task->taskNumerator() .  "/" .
    $task->taskDenominator() . "\n";
}

function reverse_complete($task) {
    echo "COMPLETE: " . $task->unique() . ", " . $task->data() . "\n";
}
?>
<?php
$gmworker= new GearmanWorker();
$gmworker->addServer();

$gmworker->addFunction("reverse", "reverse_fn");

$flag = true;

while($flag){
    $flag = $gmworker->work();
}


function reverse_fn($job) {
    echo "Received job: " . $job->handle() . "\n";
    $workload = $job->workload();

    $workload_size = $job->workloadSize();
    echo "Workload: $workload ($workload_size)\n";

    # This status loop is not needed, just showing how it works

    for ($x= 0; $x < $workload_size; $x++) {
        echo "Sending status: " . ($x + 1) . "/$workload_size complete\n";

        $job->sendStatus($x+1, $workload_size);
        $job->sendData(substr($workload, $x, 1));
        sleep(1);
    }


    $result= strrev($workload);

    echo "Result: $result\n";

    # Return what we want to send back to the client.

    return $result;
}
?>
开启三个终端,第一个终端运行 worker,另外两个终端运行 client:
[root@www test]# php w-unique.php 
Received job: H:www:641
Workload: Hi! - 2012-11-08 11:01:00 (25)
Sending status: 1/25 complete
Sending status: 2/25 complete
Sending status: 3/25 complete
Sending status: 4/25 complete
Sending status: 5/25 complete
Sending status: 6/25 complete
Sending status: 7/25 complete
Sending status: 8/25 complete
Sending status: 9/25 complete
Sending status: 10/25 complete
Sending status: 11/25 complete
Sending status: 12/25 complete
Sending status: 13/25 complete
Sending status: 14/25 complete
Sending status: 15/25 complete
Sending status: 16/25 complete
Sending status: 17/25 complete
Sending status: 18/25 complete
Sending status: 19/25 complete
Sending status: 20/25 complete
Sending status: 21/25 complete
Sending status: 22/25 complete
Sending status: 23/25 complete
Sending status: 24/25 complete
Sending status: 25/25 complete
Result: 00:10:11 80-11-2102 - !iH

[root@www test]# php c-unique.php 
Start: 2012-11-08 11:01:00
STATUS: 1, H:www:641 - 1/25
STATUS: 1, H:www:641 - 2/25
STATUS: 1, H:www:641 - 3/25
STATUS: 1, H:www:641 - 4/25
STATUS: 1, H:www:641 - 5/25
STATUS: 1, H:www:641 - 6/25
STATUS: 1, H:www:641 - 7/25
STATUS: 1, H:www:641 - 8/25
STATUS: 1, H:www:641 - 9/25
STATUS: 1, H:www:641 - 10/25
STATUS: 1, H:www:641 - 11/25
STATUS: 1, H:www:641 - 12/25
STATUS: 1, H:www:641 - 13/25
STATUS: 1, H:www:641 - 14/25
STATUS: 1, H:www:641 - 15/25
STATUS: 1, H:www:641 - 16/25
STATUS: 1, H:www:641 - 17/25
STATUS: 1, H:www:641 - 18/25
STATUS: 1, H:www:641 - 19/25
STATUS: 1, H:www:641 - 20/25
STATUS: 1, H:www:641 - 21/25
STATUS: 1, H:www:641 - 22/25
STATUS: 1, H:www:641 - 23/25
STATUS: 1, H:www:641 - 24/25
STATUS: 1, H:www:641 - 25/25
COMPLETE: 1, 00:10:11 80-11-2102 - !iH
[root@www test]# php c-unique.php
Start: 2012-11-08 11:01:02
STATUS: 1, H:www:641 - 3/25
STATUS: 1, H:www:641 - 4/25
STATUS: 1, H:www:641 - 5/25
STATUS: 1, H:www:641 - 6/25
STATUS: 1, H:www:641 - 7/25
STATUS: 1, H:www:641 - 8/25
STATUS: 1, H:www:641 - 9/25
STATUS: 1, H:www:641 - 10/25
STATUS: 1, H:www:641 - 11/25
STATUS: 1, H:www:641 - 12/25
STATUS: 1, H:www:641 - 13/25
STATUS: 1, H:www:641 - 14/25
STATUS: 1, H:www:641 - 15/25
STATUS: 1, H:www:641 - 16/25
STATUS: 1, H:www:641 - 17/25
STATUS: 1, H:www:641 - 18/25
STATUS: 1, H:www:641 - 19/25
STATUS: 1, H:www:641 - 20/25
STATUS: 1, H:www:641 - 21/25
STATUS: 1, H:www:641 - 22/25
STATUS: 1, H:www:641 - 23/25
STATUS: 1, H:www:641 - 24/25
STATUS: 1, H:www:641 - 25/25
COMPLETE: 1, 00:10:11 80-11-2102 - !iH
上面是三个终端运行的结果。terminal-2, terminal-3 提交 Job 到 Gearmand Server,terminal-1 做为 worker,执行提交到 Geramand Server 所有为 reverse 的 Job。terminal-2, terminal-3 开始时间不同,但结束时间相同。 Unique ID 相同,先前提交的 Job 没有执行完时,Gearmand 会将同 Unique Key 的 Job 合并,即它们会同时收到相同的执行情况信息:
  1. terminal-2 提交 Job A , worker 接到调度信息立即对 Job A 进行处理。
  2. 在 termial-3 提交相同 Unique Key 的 Job B,两者提交的时差为 2s。
  3. 因为 reverse worker 的运行时间 >= 25s ,所以 Job B 提交时 Job A 实际还处于执行,Gearmand Server 将 Job B 合到 Job A 一起。
  4. terminal-3 收到这时和 terminal-2 相同的 Job Status。
简单理解的话,Gearmand Server 会将 Unique Key 做为每一个 Job 的标识,当提交的 Job 的 Unique Key 相同时,服务器将其第 N (N > 1)个提交的 Job 视为重复提交,直接忽略,不对其进行任何处理。这些逻辑都只针对于当前在队列中的 Job,已经执行完毕的 Job 不会对其进行这样的处理。
测试 2
<?php
$gmc= new GearmanClient();
$gmc->addServer();

for ($i = 0; $i < 11; $i++) {
    $gmc->doBackground("reverse", "Hi! - $i", 1);
}

for ($i = 0; $i < 11; $i++) {
    $gmc->addTaskBackground("reverse", "Hi! - $i", null, 2);
}


if (! $gmc->runTasks()) {
    echo "ERROR " . $gmc->error() . "\n";
    exit;
}
?>
修改 Client,将提交的 Jobs/Tasks 改为是后台处理。假设 Gearmand Server 采用了 MySQL 的持久存储,不启动 reverse worker,只运行上述的 Client 脚本,查询数据库:
mysql> SELECT * FROM gearman_queue;
+------------+---------------+----------+----------+-------------+
| unique_key | function_name | priority | data     | when_to_run |
+------------+---------------+----------+----------+-------------+
| 1          | reverse       |        1 | Hi! - 0  |           0 |
| 2          | reverse       |        1 | Hi! - 10 |           0 |
+------------+---------------+----------+----------+-------------+
2 rows in set (0.04 sec)
从上面得知,尽管 Unqiue Key 为 12 的都提交了 10 次,而队列中对应的记录却都只有一条。也就说明,当 Unique Key 相同时,Gearmand Server 对提交的 Job/Tasks 做了合并处理。
说明:
  • 当然,从Gearmand 队列持久化其表结构中可知,unique_key 做为主键,必须是唯一的,而该值对应 Job 提交时的 Unqiue Key。即在 Unique Key 相同时,在数据库中,记录必定无法添加,即上述 例 1 所要说明的原因 (这里实际上还有点疑问, MySQL/Drizzle 和 sqlite3 的表结构竟然不同,MySQL/Drizzle 表结构竟然是在 unique_key 和 function_name 上做唯一的索引,而 unqiue_key 也没做主键,难道实际中有同 unique_key 而不同 function_name 这样的用法?而且从 MySQL 性能方面来讲,不设置主键似乎也说不过去;如果 engine 采用 InnoDB,内部还是会给建一主键。)。
    CREATE TABLE gearman_queue (    
    unique_key VARCHAR(64),    
    function_name VARCHAR(255),    
    priority INT,    
    data LONGBLOB,    
    when_to_run INT,    
    unique key (unique_key, function_name))    
    
    CREATE TABLE gearman_queue (
    unique_key TEXT PRIMARY KEY,
    function_name TEXT,
    priority INTEGER,
    data BLOB,
    when_to_run INTEGER)
    
  • 在同一个进程里,addTask/addTaskBackground 提交 Tasks 时,因为都是在调用 $gmc->runTasks() 后,所有这个进程里需提交的 Tasks 才提交到 Gearmand Server。而在提交时,如果所有 Tasks 的 Unique Key 相同,后面 Task 调用的参数数据会覆盖前面调用的。 所以,也就有上述查询中 unique_key 2 时处理的数据是 Hi! - 10

Gearman MySQL UDFs

Install


下载 Gearman User Defined Functions for MySQL 最新版。
[bash]
[root@www gearman-mysql-udf-0.6]# ./configure --libdir=/usr/local/mysql/lib/mysql/plugin/ --with-libgearman-prefix=/usr/local/gearmand/ --with-mysql=/usr/local/mysql/bin/mysql_config
[root@www gearman-mysql-udf-0.6]# make
[root@www gearman-mysql-udf-0.6]# make install
[/bash]

说明:

  • 如果 MySQL 是 source code 安装的话,必须使其支持 plugin (加上 --enable-shared配置项,且确保没有 --with-mysqld-ldflags=-all-static配置项),否则在后面步骤中创建函数时,会提示创建失败的提示:
    [sql]
    mysql> CREATE FUNCTION gman_do RETURNS STRING
    -> SONAME "libgearman_mysql_udf.so";
    ERROR 1126 (HY000): Can't open shared library 'libgearman_mysql_udf.so' (errno: 0 feature disabled)
    [/sql]

  • --libdir 确保指向 MySQL server plugin 的目录,即 MySQL plugin_dir 的值。
    [sql]
    mysql> show variables like '%plugin%';
    +---------------+-----------------------------------+
    | Variable_name | Value |
    +---------------+-----------------------------------+
    | plugin_dir | /usr/local/mysql/lib/mysql/plugin |
    +---------------+-----------------------------------+
    1 row in set (0.00 sec)
    [/sql]

  • 如果 Gearman 不是采用默认配置选项安装的话,还需用 --with-libgearman-prefix 指定 Gearman 安装目录。



安装成功的话,在 MySQL plugin 的目录中,应该有如下文件:
[bash]
[root@www gearman-mysql-udf-0.6]# ls -l /usr/local/mysql/lib/mysql/plugin/
total 128
-rwxr-xr-x 1 root root 1059 Oct 31 11:36 libgearman_mysql_udf.la
lrwxrwxrwx 1 root root 29 Oct 31 11:36 libgearman_mysql_udf.so -> libgearman_mysql_udf.so.0.0.0
lrwxrwxrwx 1 root root 29 Oct 31 11:36 libgearman_mysql_udf.so.0 -> libgearman_mysql_udf.so.0.0.0
-rwxr-xr-x 1 root root 111710 Oct 31 11:36 libgearman_mysql_udf.so.0.0.0
[/bash]

修改 plugin 目录的属主和访问权限,登录 MySQL,执行下面查询 :
[sql]
CREATE FUNCTION gman_do RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_do_high RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_do_low RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_do_background RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_do_high_background RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_do_low_background RETURNS STRING
SONAME "libgearman_mysql_udf.so";
CREATE AGGREGATE FUNCTION gman_sum RETURNS INTEGER
SONAME "libgearman_mysql_udf.so";
CREATE FUNCTION gman_servers_set RETURNS STRING
SONAME "libgearman_mysql_udf.so";
[/sql]

Run Job


函数创建完后,调用 gman_server_set 来设置 Gearman 的服务器。
[sql]
mysql> select gman_servers_set('127.0.0.1:4730,127.0.0.1:4731');
+---------------------------------------------------+
| gman_servers_set('127.0.0.1:4730,127.0.0.1:4731') |
+---------------------------------------------------+
| 127.0.0.1:4730,127.0.0.1:4731 |
+---------------------------------------------------+
1 row in set (0.00 sec)
[/sql]

也可以每个函数设置不同的 Gearman Server,如:
[sql]
mysql> select gman_servers_set('127.0.0.1', 'reverse');
+------------------------------------------+
| gman_servers_set('127.0.0.1', 'reverse') |
+------------------------------------------+
| 127.0.0.1 |
+------------------------------------------+
1 row in set (0.00 sec)

mysql> select gman_servers_set('127.0.0.1:4731', 'wc');
+------------------------------------------+
| gman_servers_set('127.0.0.1:4731', 'wc') |
+------------------------------------------+
| 127.0.0.1:4731 |
+------------------------------------------+
1 row in set (0.00 sec)
[/sql]

服务器设置好后,接着通过查询来执行 JOB:
[sql]
mysql> SELECT gman_do('reverse', host) AS test FROM mysql.user;
+----------------+
| test |
+----------------+
| 1: % |
| 2: 1.0.0.721 |
| 3: 1.0.0.721 | |
| 4: tsohlacol |
| 5: tsohlacol |
+----------------+
5 rows in set (2 min 8.43 sec)

mysql> SELECT gman_do('reverse', 'hello, world!');
+-------------------------------------+
| gman_do('reverse', 'hello, world!') |
+-------------------------------------+
| 8: !dlrow ,olleh |
+-------------------------------------+
1 row in set (0.00 sec)
[/sql]


当查询调用的是 gman_do_background() 时,返回的是 job handle,可用其来查询 job 的状态。
[sql]
mysql> SELECT gman_do_background('reverse', 'test1, test2, test3, test4');
+-------------------------------------------------------------+
| gman_do_background('reverse', 'test1, test2, test3, test4') |
+-------------------------------------------------------------+
| H:www:869425 |
+-------------------------------------------------------------+
1 row in set (0.00 sec)
[/sql]

上面例题正常进行的话,除了要启动 JOB Server gearmand外,还需要运行对应的 worker。

gman_do* 开头的函数还提供可选的第三个参数,它们用于做 unique JOB ID。在同时提交多个 JOB 给相同的 unique ID 时,Gearmand 将会在队列中把它们合并到一起,然后一次执行。需要注意的是,Gearmand 仅合并当前在队列中等待/运行 的 JOB,对于已经运行结束的 JOB 的 unique ID ,不会做跟踪处理。


Remove


如果需要移除上述创建的 MySQL UDF,执行下面查询即可:
[sql]
DROP FUNCTION gman_do;
DROP FUNCTION gman_do_high;
DROP FUNCTION gman_do_low;
DROP FUNCTION gman_do_background;
DROP FUNCTION gman_do_high_background;
DROP FUNCTION gman_do_low_background;
DROP FUNCTION gman_sum;
DROP FUNCTION gman_servers_set;
[/sql]

References:
mysql_udf_readme

How To Get System Info In Linux

FROM: How To Get System Info In Linux

General System Info


Note: 大部分获取硬件信息的命令都必须以 root 身份运行,而那些可以以其他身份运行的命令,在以 root 身份运行时,系统会提供更多的信息。


  1. 运行 qtpartedGParted 获取硬盘和分区信息。也可以用 KDiskFreefdisk -l 查看所有的分区信息。

  2. hardinfo,收集系统和硬件信息,生成报告。
    在 Synaptic (Ubuntu 桌面系统包管理器) 包管理器中,hardinfo 是安装 installation-report 包。完成后,在终端窗口运行:
    [bash gutter="false"]
    report-hw
    # 或将报告生成名为 hwreport, 存入 $HOME 目录下的 text 文件
    report-hw > hwreport
    [/bash]

  3. sysinfo 在终端运行,以图形方式展现当前系统的所有统计信息。

  4. hal-device-manager,kde-hal-device-manager 可通过 Synaptic 安装,即 KMenu > System > Device Manager > kde-hal-device-manager。



    • dmidecode -t memory (以 root 身份) dmidecode 工具将导出系统 DMI(Desktop Management Interface) 表。这个表包括系统硬件组件,BIOS 版本信息等。它不仅包括系统当前的配置信息,还包括可支持的 CPU 速度下的 BIOS 限制,允许的最大内存等。

    • dmidecode | less (以 root 身份) 以分页形式查看 BIOS 和 系统信息 (按 space 往下查看,按 q 退出)





    • uname -a 打印详细信息,依次为内核名称,主机名,内核版本号,内核版本,硬件名,处理器类型,硬件平台类型,操作系统名称。

    • uname -m 打印主机硬件名。

    • uname -r 内核版本。


    [bash]
    [root@www ~]# uname -a
    Linux wwww 2.6.27 #1 SMP Fri Oct 15 00:59:47 CST 2010 x86_64 x86_64 x86_64 GNU/Linux
    [root@www ~]# uname -m
    x86_64
    [root@www ~]# uname -r
    2.6.27
    [/bash]



    • lshw (以 root 用户) 以纯文本的格式输出硬件报告,也可生成 html、xml 等格式的报告。可用 lshw | less 分页查看 (space 往下参看, q 退出)

    • lshw-gtk (以 root 用户) 可从 Synaptic 下载安装。在图形界面下查看硬件信息。



  5. lsb_release -a 或查看 /etc/lsb_release.d
    [bash]
    [root@www ~]# lsb_release -a
    LSB Version: :core-3.1-amd64:core-3.1-ia32:core-3.1-noarch:graphics-3.1-amd64:graphics-3.1-ia32:graphics-3.1-noarch
    Distributor ID: CentOS
    Description: CentOS release 5.3 (Final)
    Release: 5.3
    Codename: Final
    [/bash]
    Mepis Linux 6.5 上,登录界面也可查看版本信息。



    • lspci (以 root 身份) 查看 PCI 设备, -v 详细信息, -vv 更多详细信息。

    • lspci -tv (以 root 身份) 以树状形式展示。

    • lsusblsusb -tv 查看 USB 设备的详细信息。

    • lsmod (以 root 身份) 查看目前内核加载的模块,也可用 lsmod | less 分页展示。


    [bash collapse="true"]
    [root@www ~]# lspci
    00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 13)
    00:01.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 1 (rev 13)
    00:03.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 3 (rev 13)
    00:07.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 7 (rev 13)
    00:14.0 PIC: Intel Corporation X58 I/O Hub System Management Registers (rev 13)
    00:14.1 PIC: Intel Corporation X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
    00:14.2 PIC: Intel Corporation X58 I/O Hub Control Status and RAS Registers (rev 13)
    00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
    00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
    00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
    00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
    00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
    00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
    00:1d.3 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
    00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
    00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
    00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
    00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller
    00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller
    01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
    01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
    02:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
    04:03.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200eW WPCM450 (rev 0a)
    [root@www ~]# lspci -tv
    -[0000:00]-+-00.0 Intel Corporation X58 I/O Hub to ESI Port
    +-01.0-[0000:01]--+-00.0 Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet
    | \-00.1 Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet
    +-03.0-[0000:02]----00.0 LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS
    +-07.0-[0000:03]--
    +-14.0 Intel Corporation X58 I/O Hub System Management Registers
    +-14.1 Intel Corporation X58 I/O Hub GPIO and Scratch Pad Registers
    +-14.2 Intel Corporation X58 I/O Hub Control Status and RAS Registers
    +-1a.0 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
    +-1a.1 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
    +-1a.7 Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
    +-1d.0 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
    +-1d.1 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
    +-1d.2 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
    +-1d.3 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
    +-1d.7 Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
    +-1e.0-[0000:04]----03.0 Matrox Graphics, Inc. MGA G200eW WPCM450
    +-1f.0 Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
    +-1f.2 Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller
    \-1f.5 Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller
    [root@www ~]# lsusb
    Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp.
    Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    [root@www ~]# lsusb -tv
    Bus# 0
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 7
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 6
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 5
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 4
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 3
    `-Dev# 1 Vendor 0x1d6b Product 0x0001
    Bus# 2
    `-Dev# 1 Vendor 0x1d6b Product 0x0002
    Bus# 1
    `-Dev# 1 Vendor 0x1d6b Product 0x0002
    `-Dev# 2 Vendor 0x0424 Product 0x2514
    [root@www ~]#
    [root@www ~]# lsmod
    Module Size Used by
    nfsd 262312 17
    auth_rpcgss 55968 1 nfsd
    exportfs 20928 1 nfsd
    nfs 266032 1
    nfs_acl 19840 2 nfsd,nfs
    lockd 80784 2 nfsd,nfs
    autofs4 36104 2
    hidp 31296 2
    rfcomm 50912 0
    l2cap 36864 10 hidp,rfcomm
    bluetooth 73764 5 hidp,rfcomm,l2cap
    sunrpc 210056 17 nfsd,auth_rpcgss,nfs,nfs_acl,lockd
    ipv6 273320 94
    dm_mirror 32640 0
    dm_log 26244 1 dm_mirror
    dm_multipath 32344 0
    scsi_dh 23748 1 dm_multipath
    dm_mod 71408 3 dm_mirror,dm_log,dm_multipath
    sbs 30160 0
    sbshc 22720 1 sbs
    battery 29192 0
    acpi_memhotplug 22340 0
    ac 21640 0
    parport_pc 43496 0
    lp 27844 0
    parport 53872 2 parport_pc,lp
    sg 48864 0
    dcdbas 24240 0
    sr_mod 31748 0
    cdrom 49768 1 sr_mod
    serio_raw 22276 0
    button 23968 0
    rtc_cmos 26424 0
    rtc_core 33348 1 rtc_cmos
    rtc_lib 19776 1 rtc_core
    bnx2 186824 0
    pcspkr 19328 0
    ata_piix 36484 0
    libata 181504 1 ata_piix
    shpchp 45468 0
    mptsas 47696 7
    mptscsih 49472 1 mptsas
    mptbase 90020 2 mptsas,mptscsih
    scsi_transport_sas 47616 1 mptsas
    sd_mod 42984 11
    scsi_mod 172536 8 scsi_dh,sg,sr_mod,libata,mptsas,mptscsih,scsi_transport_sas,sd_mod
    ext3 136784 2
    jbd 62824 1 ext3
    uhci_hcd 37408 0
    ohci_hcd 38172 0
    ehci_hcd 47948 0
    [/bash]



  6. /proc 这个目录下有很多重要的信息。cd /procls,用 cat 读取该目录下的信息,如 cpuinfo, devices, filesystems, meminfo, partitions, swaps, uptime, version 等,也可进 proc 的子目录查看信息,如 cd driver
    [bash collapse="true"]
    [root@www ~]# cat /proc/cpuinfo
    processor : 0
    vendor_id : GenuineIntel
    cpu family : 6
    model : 26
    model name : Intel(R) Xeon(R) CPU E5502 @ 1.87GHz
    stepping : 5
    cpu MHz : 1861.992
    cache size : 4096 KB
    physical id : 1
    siblings : 2
    core id : 0
    cpu cores : 2
    apicid : 16
    initial apicid : 16
    fpu : yes
    fpu_exception : yes
    cpuid level : 11
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca sse4_1 sse4_2 popcnt lahf_lm
    bogomips : 3723.98
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management:

    processor : 1
    vendor_id : GenuineIntel
    cpu family : 6
    model : 26
    model name : Intel(R) Xeon(R) CPU E5502 @ 1.87GHz
    stepping : 5
    cpu MHz : 1861.992
    cache size : 4096 KB
    physical id : 0
    siblings : 2
    core id : 0
    cpu cores : 2
    apicid : 0
    initial apicid : 0
    fpu : yes
    fpu_exception : yes
    cpuid level : 11
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca sse4_1 sse4_2 popcnt lahf_lm
    bogomips : 3723.44
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management:

    processor : 2
    vendor_id : GenuineIntel
    cpu family : 6
    model : 26
    model name : Intel(R) Xeon(R) CPU E5502 @ 1.87GHz
    stepping : 5
    cpu MHz : 1861.992
    cache size : 4096 KB
    physical id : 1
    siblings : 2
    core id : 2
    cpu cores : 2
    apicid : 20
    initial apicid : 20
    fpu : yes
    fpu_exception : yes
    cpuid level : 11
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca sse4_1 sse4_2 popcnt lahf_lm
    bogomips : 3723.42
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management:

    processor : 3
    vendor_id : GenuineIntel
    cpu family : 6
    model : 26
    model name : Intel(R) Xeon(R) CPU E5502 @ 1.87GHz
    stepping : 5
    cpu MHz : 1861.992
    cache size : 4096 KB
    physical id : 0
    siblings : 2
    core id : 2
    cpu cores : 2
    apicid : 4
    initial apicid : 4
    fpu : yes
    fpu_exception : yes
    cpuid level : 11
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca sse4_1 sse4_2 popcnt lahf_lm
    bogomips : 3723.44
    clflush size : 64
    cache_alignment : 64
    address sizes : 40 bits physical, 48 bits virtual
    power management:

    [root@www ~]# cat /proc/version
    Linux version 2.6.27 (root@www) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Fri Oct 15 00:59:47 CST 2010
    [root@www ~]# cat /proc/swaps
    Filename Type Size Used Priority
    /dev/sda3 partition 4192956 212 -1
    [/bash]


  7. top 实时显示系统当前的进程和其他状况,详细到 这里
    [bash collapse="true"]
    [root@www ~]# top
    top - 16:36:08 up 217 days, 1:55, 1 user, load average: 0.00, 0.00, 0.00
    Tasks: 182 total, 1 running, 181 sleeping, 0 stopped, 0 zombie
    Cpu0 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu1 : 0.0%us, 0.0%sy, 0.0%ni, 99.0%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu2 : 0.0%us, 0.3%sy, 0.0%ni, 98.7%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Cpu3 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 8186612k total, 8021152k used, 165460k free, 201404k buffers
    Swap: 4192956k total, 212k used, 4192744k free, 6758848k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    13373 root 20 0 1165m 82m 9924 S 0.3 1.0 17:58.03 java
    24688 root 20 0 2474m 665m 651m S 0.3 8.3 31344:13 mongod
    1 root 20 0 10356 640 536 S 0.0 0.0 1:06.25 init
    2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd
    3 root RT -5 0 0 0 S 0.0 0.0 0:00.60 migration/0 4 root 15 -5 0 0 0 S 0.0 0.0 0:01.02 ksoftirqd/0 5 root RT -5 0 0 0 S 0.0 0.0 0:00.73 watchdog/0
    6 root RT -5 0 0 0 S 0.0 0.0 0:01.13 migration/1
    7 root 15 -5 0 0 0 S 0.0 0.0 0:00.77 ksoftirqd/1
    8 root RT -5 0 0 0 S 0.0 0.0 0:00.03 watchdog/1
    9 root RT -5 0 0 0 S 0.0 0.0 0:01.89 migration/2
    10 root 15 -5 0 0 0 S 0.0 0.0 0:00.57 ksoftirqd/2
    11 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
    12 root RT -5 0 0 0 S 0.0 0.0 0:02.08 migration/3
    13 root 15 -5 0 0 0 S 0.0 0.0 0:00.63 ksoftirqd/3
    14 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
    15 root 15 -5 0 0 0 S 0.0 0.0 4:57.46 events/0
    16 root 15 -5 0 0 0 S 0.0 0.0 3:36.12 events/1
    17 root 15 -5 0 0 0 S 0.0 0.0 3:52.82 events/2
    18 root 15 -5 0 0 0 S 0.0 0.0 3:43.96 events/3
    19 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
    141 root 15 -5 0 0 0 S 0.0 0.0 0:01.16 kblockd/0
    142 root 15 -5 0 0 0 S 0.0 0.0 0:00.43 kblockd/1
    143 root 15 -5 0 0 0 S 0.0 0.0 0:07.63 kblockd/2
    144 root 15 -5 0 0 0 S 0.0 0.0 0:00.56 kblockd/3
    146 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
    147 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpi_notify
    256 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue
    258 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 ksuspend_usbd
    264 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khubd
    267 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kseriod
    340 root 15 -5 0 0 0 S 0.0 0.0 0:17.71 kswapd0
    341 root 15 -5 0 0 0 S 0.0 0.0 0:21.69 kswapd1
    342 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0
    343 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/1
    344 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/2
    345 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 aio/3
    [/bash]

  8. htoptop 更好,可查看进程间的交互。

  9. ctrl-es KDE 才有,按下后出现 Process Table。

  10. prcinfo 收集 /proc 下的信息并将之打印。

  11. 其它相关命令

    • ps aux

    • ps -e

    • swapon -s 显示 swap 的分区和信息
      [bash]
      [root@www ~]# swapon -s
      Filename Type Size Used Priority
      /dev/sda3 partition 4192956 212 -1
      [/bash]

    • swapon -a 自动启动所有 swap

    • /etc/fstab 包含文件系统的配置信息
      [bash collapse="true"]
      [root@www ~]# cat /etc/fstab
      LABEL=/ / ext3 defaults 1 1
      LABEL=/boot1 /boot ext3 defaults 1 2
      tmpfs /dev/shm tmpfs defaults 0 0
      devpts /dev/pts devpts gid=5,mode=620 0 0
      sysfs /sys sysfs defaults 0 0
      proc /proc proc defaults 0 0
      LABEL=SWAP-sda3 swap swap defaults 0 0
      /dev/sdb1 /data/dev10 xfs defaults 0 0
      /dev/sdc1 /data/dev11 xfs defaults 0 0
      /dev/sdd1 /data/dev12 xfs defaults 0 0
      /dev/sda4 /soft xfs defaults 0 0
      [/bash]

    • free 查看内存使用情况,详细 这里

    • KSysGuard 系统监视器,即 KDE 系统监视器,当中有包含内存的使用情况报告。

    • df -h , 以易读的方式显示目前磁盘空间和使用情况,详细 这里

    • du / -bh | more

    • du -s /var/log/*

    • hdparm -t /dev/sda 显示硬盘 sda 当前性能状况。
      [bash]
      [root@www ~]# hdparm -t /dev/sda

      /dev/sda:
      Timing buffered disk reads: 448 MB in 3.01 seconds = 148.74 MB/sec
      You have new mail in /var/spool/mail/root
      [/bash]

    • arch 显示 CPU 的标志,但这个可能错误,如果有安装 rpm,可用 rpm --showrc | grep " arch" 检测。
      [bash]
      [root@www ~]# arch
      x86_64
      [root@www ~]# rpm --showrc | grep " arch"
      build arch : x86_64
      compatible build archs: ia32e x86_64 noarch
      install arch : ia32e
      compatible archs : ia32e x86_64 athlon noarch amd64 i686 i586 i486 i386 fat
      [/bash]

    • ifconfig -a

    • date

    • uptime



  12. cat /proc/PID/staus 查看进程当前状态
    [bash collapse="true"]
    [root@www ~]# cat /proc/12452/status
    Name: gearmand
    State: S (sleeping)
    Tgid: 12452
    Pid: 12452
    PPid: 1
    TracerPid: 0
    Uid: 0 0 0 0
    Gid: 0 0 0 0
    FDSize: 64
    Groups: 0 1 2 3 4 6 10
    VmPeak: 407808 kB
    VmSize: 309188 kB
    VmLck: 0 kB
    VmHWM: 232476 kB
    VmRSS: 109976 kB
    VmData: 275676 kB
    VmStk: 84 kB
    VmExe: 284 kB
    VmLib: 4172 kB
    VmPTE: 404 kB
    Threads: 7
    SigQ: 1/71680
    SigPnd: 0000000000000000
    ShdPnd: 0000000000000000
    SigBlk: 0000000000000000
    SigIgn: 0000000000001000
    SigCgt: 0000000180004203
    CapInh: 0000000000000000
    CapPrm: fffffffffffffeff
    CapEff: fffffffffffffeff
    CapBnd: fffffffffffffeff
    voluntary_ctxt_switches: 48135
    nonvoluntary_ctxt_switches: 19085
    [/bash]



Bash Info



  1. echo $PATH 输出当前用户环境变量 PATH

  2. echo $? 输出最后一次调用的命令的退出码。

  3. help 列出所有 shell 内置的命名, help name 显示命令 name 的帮助信息。

  4. whereis 列出命令的二进制文件,源代码,以及帮助文件的位置。

  5. which

  6. type (name) 表明当 name 做为命令执行时的解析情况

  7. alias 列出所有的别名命令

  8. unalias 永久的删除所有别名命令,直到重新设置。

  9. "command" 命令用引号引起来,将会做为 true 命令调用 (如果存在的话),而不是其别名。

  10. file 显示文件类型
    [bash]
    [root@www tmp]# file test.php
    test.php: PHP script text
    [root@www tmp]# file -i test.php
    test.php: text/plain; charset=us-ascii
    [root@www tmp]# file -iz test.php
    test.php: text/plain; charset=us-ascii
    [root@www tmp]# file -s test.php
    test.php: PHP script text
    [/bash]

  11. tree
    [bash]
    [root@www test]# tree
    .
    |-- 1.php
    |-- 2.php
    `-- 3.php

    0 directories, 3 files
    [/bash]

  12. cal
    [bash collapse="true"]
    [root@www ~]# cal
    October 2012
    Su Mo Tu We Th Fr Sa
    1 2 3 4 5 6
    7 8 9 10 11 12 13
    14 15 16 17 18 19 20
    21 22 23 24 25 26 27
    28 29 30 31
    [root@www test]# cal -3
    September 2012 October 2012 November 2012
    Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa
    1 1 2 3 4 5 6 1 2 3
    2 3 4 5 6 7 8 7 8 9 10 11 12 13 4 5 6 7 8 9 10
    9 10 11 12 13 14 15 14 15 16 17 18 19 20 11 12 13 14 15 16 17
    16 17 18 19 20 21 22 21 22 23 24 25 26 27 18 19 20 21 22 23 24
    23 24 25 26 27 28 29 28 29 30 31 25 26 27 28 29 30
    30
    [root@www test]# cal 10 2012
    October 2012
    Su Mo Tu We Th Fr Sa
    1 2 3 4 5 6
    7 8 9 10 11 12 13
    14 15 16 17 18 19 20
    21 22 23 24 25 26 27
    28 29 30 31

    [/bash]

Install systemtap

CentOS 5.3, Linux 2.6.6,64 位

下载 源代码 ,配置,报错:
[bash]
[root@www systemtap-2.0]# ./configure --prefix=/usr/local/systemtap
......
configure: error: missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel)
[/bash]

据错误提示,用 yum 安装依赖:
[bash]
[root@www systemtap-2.0]# yum install elfutils.x86_64 -y
[root@www systemtap-2.0]# yum install elfutils-devel.x86_64 -y
[/bash]

配置 systemtap,提示:
[bash]
[root@www systemtap-2.0]# ./configure --prefix=/usr/local/systemtap
......
configure: error: elfutils, libdw too old, need 0.148+
[/bash]

Googling elfutils, libdw,基本都是 rpm 包,于是换成用 rpm 包来安装,从 这里 下载 systemtapkernel-develsystemtap-runtime 这三个 rpm 包,system-runtime 和 kernel-devel 包是 systemtap 的依赖包。

[bash]
[root@www src]# rpm -Uhv systemtap-1.6-7.el5_8.x86_64.rpm systemtap-runtime-1.6-7.el5_8.x86_64.rpm kernel-devel-2.6.18-308.el5.x86_64.rpm
Preparing... ########################################### [100%]
package kernel-devel-2.6.18-308.el5.x86_64 is already installed
package systemtap-runtime-1.6-7.el5_8.x86_64 is already installed
package systemtap-1.6-7.el5_8.x86_64 is already installed
# 检查安装是否成功。如果部署成功,运行下面命令,输出应该类似于如下:
[root@www src]# stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 76 library script(s) using 147620virt/22500res/2996shr kb, in 190usr/20sys/240real ms.
Pass 2: analyzed script: 1 probe(s), 22 function(s), 3 embed(s), 1 global(s) using 277912virt/85372res/5892shr kb, in 1140usr/90sys/2714real ms.
Pass 3: translated to C into "/tmp/stapXXnotG/stap_59817f232930c1e1240b1253994e85c9_10707_src.c" using 272140virt/84864res/6580shr kb, in 380usr/20sys/409real ms.
Pass 4: compiled C into "stap_59817f232930c1e1240b1253994e85c9_10707.ko" in 2060usr/290sys/3607real ms.
Pass 5: starting run.
read performed
Pass 5: run completed in 10usr/20sys/557real ms.
[/bash]

Resourcess: