標籤

4GL (1) 人才發展 (10) 人物 (3) 太陽能 (4) 心理 (3) 心靈 (10) 文學 (31) 生活常識 (14) 光學 (1) 名句 (10) 即時通訊軟體 (2) 奇狐 (2) 音樂 (2) 產業 (5) 郭語錄 (3) 無聊 (3) 統計 (4) 新聞 (1) 經濟學 (1) 經營管理 (42) 解析度 (1) 遊戲 (5) 電學 (1) 網管 (10) 廣告 (1) 數學 (1) 機率 (1) 雜趣 (1) 證券 (4) 證券期貨 (1) ABAP (15) AD (1) agentflow (4) AJAX (1) Android (1) AnyChart (1) Apache (14) BASIS (4) BDL (1) C# (1) Church (1) CIE (1) CO (38) Converter (1) cron (1) CSS (23) DMS (1) DVD (1) Eclipse (1) English (1) excel (5) Exchange (4) Failover (1) FI (57) File Transfer (1) Firefox (2) FM (2) fourjs (1) gladiatus (1) google (1) Google Maps API (2) grep (1) Grub (1) HR (2) html (23) HTS (8) IE (1) IE 8 (1) IIS (1) IMAP (3) Internet Explorer (1) java (3) JavaScript (22) jQuery (6) JSON (1) K3b (1) LED (3) Linux (112) Linux Mint (4) Load Balance (1) Microsoft (2) MIS (2) MM (51) MSSQL (1) MySQL (27) Network (1) NFS (1) Office (1) Oracle (125) Outlook (3) PDF (6) Perl (59) PHP (33) PL/SQL (1) PL/SQL Developer (1) PM (3) Postfix (2) postfwd (1) PostgreSQL (1) PP (50) python (1) QM (1) Red Hat (4) Reporting Service (28) ruby (11) SAP (234) scp (1) SD (16) sed (1) Selenium-WebDriver (5) shell (5) SQL (4) SQL server (8) SQuirreL SQL Client (1) SSH (2) SWOT (3) Symantec (2) T-SQL (7) Tera Term (2) tip (1) tiptop (22) Tomcat (6) Trouble Shooting (1) Tuning (5) Ubuntu (33) ufw (1) utf-8 (1) VIM (11) Virtual Machine (2) vnc (3) Web Service (2) wget (1) Windows (19) Windows (1) WM (6) youtube (1) yum (2)

2012年6月24日 星期日

Oracle : exp/imp with named pipe

Oracle的exp/imp是许多用户经常使用的两个工具. 它们常被用来做数据库的逻辑备份,数据库重组和数据转移等工作.
先由exp把数据卸出到文件系统, 产生一个.dmp文件, 然后必要时再由imp将数据装入数据库. 对于一般中小型数据库来说, 全数据库的exp所产生的dmp文件可能小于2GB, 但对稍大型的数据库, exp产生的数据动辄数十至上百个GB. 而现时多数操作系统为32位, 其文件系统允许的最大文件为2GB. 这样显然不能由文件系统存放exp产生的数据. 这是问题之一. 另一个问题是随着数据库的不断增大, exp所需时间越来越长以致实际上很难实施. 本文针对以上两个问题讨论相应对策.

我们以UNIX为例首先看看怎么样超越2GB限制. 这需利用UNIX的通用技术, 如管道(named pipe), 数据拷贝工具dd以及数据压缩(compress). 下面分别讨论这几种技术.

·管道 是一种伪文件. 它存在于内存中, 用于快速I/O操作. 管道的缓冲区采用先进先出机制, 即写管道进程写到缓冲区头部而读管道进程读取管道尾部. 建立管道的命令为”mknod filename p".
·dd 允许我们从一个设备拷贝数据到另一个设备.
·compress 为UNIX数据压缩工具.
实施exp之前, 我们可先检查所产生dmp文件的大小. 以下步骤既可实现,
1. 生成管道:
$ mknod /tmp/mypipe p
2. exp结果输出到该管道
$ exp file=/tmp/mypipe <...其它exp选项> &
3. 读取管道并把结果输出到管道, 只为查看数据量:
$ dd if=/tmp/mypipe of=/dev/null
结果返回exp (.dmp)文件大小, 单位为块(512 bytes).
现在我们可以着手讨论具体方法,
1. 文件压缩
2. 直接exp到磁带
3. 综合方法
首先看文件压缩方法. 利用文件压缩我们可尽量避免dmp文件大小超过2GB限制. 我们利用UNIX的管道技术, 具体步骤如下,
1. 启动compress进程, 使之从管道读取数据并输出到磁盘文件.
$ mknod /tmp/exp_pipe p
$ compress < /tmp/exp_pipe > export.dmp.Z &
2. exp到管道.
$ exp file=/tmp/exp_pipe ... &
imp时情况类似.
再看看直接exp到磁带上的方法.
$ exp file=/dev/rmt/0m volsize=4G
$ imp file=/dev/rmt/0m volsize=4G
最后看看exp到裸设备上的方法,
$ exp file=/dev/rdsk/c0t3d0s0 volsize=4G
imp类似.以上方法都可使我们避免2GB限制.
下 面讨论怎么样缩短exp索需时间的问题. 目前多数用户会采用exp到磁带的方法只因没有足够的剩余硬盘空间. 而整个exp过程所花时间的大部分都在写磁带上. 通过我们上面对UNIX工具的讨论, 我们可以做到先exp到管道, 再将管道数据压缩后输出到磁带上. 这样写磁带的数据量可大大减少, 从而在相当程度上缩短写磁带时间. 另外从Oracle内部角度讲, 从7.3版开始, Oracle允许用户做direct path export, 即跳过Oracle
$ make -f $ORACLE_HOME/rdbms/lib/oracle.mk expst
$ make -f $ORACLE_HOME/rdbms/lib/oracle.mk impst
以 上步骤产生的可执行文件expst (exp single task)和impst与exp/imp用法完全相同. 使用它们比使用exp/imp可节省多至30%的时间. 但在使用它们时千万要注意一点, 即一定要保证在使用expst/impst时不能有任何其他用户在使用Oracle数据库, 否则数据库会产生不可预知的后果甚至毁坏.
A:昨天在网上一个朋友问我,如何导入一个通过pipe导出并压缩过的文件,我特意做了一次试验:
  Oracle的导出和导入不能直接引用标准的输出和输入,但通过unix pipe可是实现,通过管道可以加快exp/imp速度
  通过管道导出数据
  1.通过mknod -p建立管道
  mknod /home/exppipe p --在目录/home下建立一个管道exppipe 注意参数p
  2.通过exp和gzip导出数据到建立的管道并压缩
  exp test/test@orcl file=/home/exppipe &
  gzip < /home/exppipe > exp.dmp.gz
  3.导出成功完成之后删除建立的管道
  rm -rf /home/exppipe
  根据我的测试,压缩导出文件只是普通大小的十分之一
  通过管道导入生成的文件
  1.建立管道
mknod /home/exppipe p
  2.导入生成的压缩文件
  imp system/passwd@orcl file=/home/exppipe fromuser=test touser=macro &
  gunzip < exp.dmp.gz > /home/exppipe
  这样就导入成功,之后删除管道
  3.删除管道
 
  rm /home/exppipe
  以上总结,PiPe真不错,能加快速度,同时还可以避免单个文件大于2G的限制。

Oracle : exp/imp 字符集轉換

ORACLE多国语言设置是为了支持世界范围的语言与字符集,一般对语言提示, 货币形式,排序方式和 CHAR,VARCHAR2,C LOB,LONG 字段的数据的显示等有效。ORACLE 的多国语言设置最主要的两个特性就是国家语言设置与字符集设置,国家语 言设置决定了界面或提示使用的语言种类,字符集决定了数据库保存与字符集有关数据(如文本)时候的编码规则。
ORACLE字符集设定,分为数据库字符集和客户端字符集环境设置。在数据库端,
字符集在创建数据库的时候设定,并保存在数据库props$表中。
在客户端的字符集环境比较简单,主要就是环境变量或注册表项 NLS_ LANG,注意 NLS_LANG的优先级别为:参数文件<注册表<环境变量<alter session.如果客户端 字符集和服务器端字符集不一样,而且字符集的转换也不兼容,那么客户端的数据显示与导出 导入的与字符集有关的数据将都是乱码。

使用一点点技巧,就可以使导出/导入在不同的字符集的数据库上转换数据。这里需要一个2进制文 件编辑工具即可,如 uedit32.用编辑方式打开导出的dmp文件,获取 2 、3 字节 的内容 , 如 00 01 , 先把它转换 为 10 进制数,为1 ,使用函数
NLS_CHARSET_NAME 即可获得该字符集:
SQL> select nls_charset_name(1) from dual; NLS_CHARSET_NAME(1)
------------------- US7ASCII
可以知道该dmp文件的字符集为 US7ASCII,如果需要把该 dmp文件的字符集换成ZHS16GBK,则需要用 NLS_CHARSET_ID 获取该字符集的编号: SQL> select nls_charset_id('zhs16gbk') from dual; NLS_CHARSET_ID('ZHS16GBK')
--------------------------
把852换成16进制数,为354,把 2、3字节的 00 01 换成03 54,即完成了把该 dmp文件字符集从us7ascii 到 zhs16gbk 的转化,这样,再把该dmp文件导入到 zhs16gbk 字符集的数据库就可以了。

2012年6月18日 星期一

URL encode a string

URL addresses only accepts alphanumeric characters and some punctuation symbols, like parenthesis and underscore.
If you need to use any other symbol (like space) you have to URL encode it using a percent sign followed by the two hexadecimal digits that represents that digit in the ASCII table.
For example, the space symbol is character 32 (hexadecimal 20) in the ASCII table, so it's expressed as %20.
In Perl, the easiest way to URL encode a string is to use uri_escape() function from URI::Escape module. This function converts all the unsafe symbols of a string to its URL encode representation.
Conversely, uri_unescape() converts a URL encoded string to its normal representation.
Example:
#!/usr/bin/perl

use URI::Escape;

my $string = "Hello world!";
my $encode = uri_escape($string);

print "Original string: $string\n";
print "URL Encoded string: $encode\n";

find + awk

find . -type f -size +200M -exec ls -lh {} \; | awk '{print $8 ":" $5}'

2012年6月12日 星期二

grep OR AND NOT 用法

Question: Can you explain how to use OR, AND and NOT operators in Unix grep command with some examples?
Answer: In grep, we have options equivalent to OR and NOT operators. There is no grep AND opearator. But, you can simulate AND using patterns. The examples mentioned below will help you to understand how to use OR, AND and NOT in Linux grep command.

The following employee.txt file is used in the following examples.
$ cat employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
400  Nisha   Manager    Marketing   $9,500
500  Randy   Manager    Sales       $6,000
You already knew that grep is extremely powerful based on these grep command examples.

Grep OR Operator

Use any one of the following 4 methods for grep OR. I prefer method number 3 mentioned below for grep OR operator.

1. Grep OR Using \|

If you use the grep command without any option, you need to use \| to separate multiple patterns for the or condition.
grep 'pattern1\|pattern2' filename
For example, grep either Tech or Sales from the employee.txt file. Without the back slash in front of the pipe, the following will not work.
$ grep 'Tech\|Sales' employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
500  Randy   Manager    Sales       $6,000

2. Grep OR Using -E

grep -E option is for extended regexp. If you use the grep command with -E option, you just need to use | to separate multiple patterns for the or condition.
grep -E 'pattern1|pattern2' filename
For example, grep either Tech or Sales from the employee.txt file. Just use the | to separate multiple OR patterns.
$ grep -E 'Tech|Sales' employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
500  Randy   Manager    Sales       $6,000

3. Grep OR Using egrep

egrep is exactly same as ‘grep -E’. So, use egrep (without any option) and separate multiple patterns for the or condition.
egrep 'pattern1|pattern2' filename
For example, grep either Tech or Sales from the employee.txt file. Just use the | to separate multiple OR patterns.
$ egrep 'Tech|Sales' employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
500  Randy   Manager    Sales       $6,000

4. Grep OR Using grep -e

Using grep -e option you can pass only one parameter. Use multiple -e option in a single command to use multiple patterns for the or condition.
grep -e pattern1 -e pattern2 filename
For example, grep either Tech or Sales from the employee.txt file. Use multiple -e option with grep for the multiple OR patterns.
$ grep -e Tech -e Sales employee.txt
100  Thomas  Manager    Sales       $5,000
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
500  Randy   Manager    Sales       $6,000

Grep AND

5. Grep AND using -E ‘pattern1.*pattern2′

There is no AND operator in grep. But, you can simulate AND using grep -E option.
grep -E 'pattern1.*pattern2' filename
grep -E 'pattern1.*pattern2|pattern2.*pattern1' filename
The following example will grep all the lines that contain both “Dev” and “Tech” in it (in the same order).
$ grep -E 'Dev.*Tech' employee.txt
200  Jason   Developer  Technology  $5,500
The following example will grep all the lines that contain both “Manager” and “Sales” in it (in any order).
$ grep -E 'Manager.*Sales|Sales.*Manager' employee.txt
Note: Using regular expressions in grep is very powerful if you know how to use it effectively.

6. Grep AND using Multiple grep command

You can also use multiple grep command separated by pipe to simulate AND scenario.
grep -E 'pattern1' filename | grep -E 'pattern2'
The following example will grep all the lines that contain both “Manager” and “Sales” in the same line.
$ grep Manager employee.txt | grep Sales
100  Thomas  Manager    Sales       $5,000
500  Randy   Manager    Sales       $6,000

Grep NOT

7. Grep NOT using grep -v

Using grep -v you can simulate the NOT conditions. -v option is for invert match. i.e It matches all the lines except the given pattern.
grep -v 'pattern1' filename
For example, display all the lines except those that contains the keyword “Sales”.
$ grep -v Sales employee.txt
200  Jason   Developer  Technology  $5,500
300  Raj     Sysadmin   Technology  $7,000
400  Nisha   Manager    Marketing   $9,500
You can also combine NOT with other operator to get some powerful combinations.
For example, the following will display either Manager or Developer (bot ignore Sales).
$ egrep 'Manager|Developer' employee.txt | grep -v Sales
200  Jason   Developer  Technology  $5,500
400  Nisha   Manager    Marketing   $9,500
 

2012年6月8日 星期五

Reporting Service : DateAdd

Syntax

          

DATEADD (datepart , number , date ) 
 
Arguments

datepart
Is the part of date to which an integer number is added. The following table lists all valid datepart arguments. User-defined variable equivalents are not valid.
datepart
Abbreviations
year
yy , yyyy
quarter
qq , q
month
mm , m
dayofyear
dy , y
day
dd , d
week
wk , ww
weekday
dw , w
hour
hh
minute
mi , n
second
ss , s
millisecond
ms
microsecond
mcs
nanosecond
ns
number
Is an expression that can be resolved to an int that is added to a datepart of date. User-defined variables are valid.
If you specify a value with a decimal fraction, the fraction is truncated and not rounded.
date
Is an expression that can be resolved to a time, date, smalldatetime, datetime, datetime2, or datetimeoffset value. date can be an expression, column expression, user-defined variable, or string literal. If the expression is a string literal, it must resolve to a datetime. To avoid ambiguity, use four-digit years. For information about two-digit years, see Configure the two digit year cutoff Server Configuration Option.
 

2012年6月1日 星期五

二战的名句

http://www.uplei.cn/post/49.html
德国神父马丁在二战犹太人蒙难纪念碑上留下一段名句:

  “起初,他们追杀共产主义者,我不是共产主义者,我不说话;

  接着,他们追杀犹太人,我不是犹太人,我不说话;

  后来,他们追杀工会成员,我不是工会成员,我不说话;

  此后,他们追杀天主教徒,我是新教徒,我不说话;

  最后,他们奔我而来,再也没有人站出来为我说话了。” 

我们是伞兵,理所当然要被包围

 

“告诉元首,我已尽力。告诉我父亲,我依然爱他!”

一位受重伤即将阵亡的德军士兵对他的战友说的遗言!